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Executive Summary 

College admissions and placement tests play a crucial role in the American education system. 
More than 2 million students each year take admissions tests (the ACT or SAT), and the 
results help postsecondary institutions make critical decisions about who will go to college, 
where they will be admitted and the likelihood of their success in a broad range of college 
courses. College placement tests, meanwhile, are used by a majority of the nation’s colleges 
and universities to determine which courses young people are prepared to enter. 



Now, as many states seek to raise high school standards 
to ensure that more students graduate prepared for the 
demands of college and the workplace, college admis- 
sions and placement tests are being called upon to serve 
new purposes for which they were not intentionally 
designed. Some states, for example, are using the SAT 
and ACT as their official statewide high school gradu- 
ation exam, incorporating these tests into their state 
assessment and accountability systems. Other states are 
considering whether college placement tests could be 
given to students while still in high school to provide 
early feedback on their level of readiness. Still others 
are developing end-of-course tests that align with their 
high school curriculum and tap college-ready content or 
modifying their existing high school tests to make them 
better measures of college readiness. 

Achieve launched this study to help inform the deci- 
sions states are making about high school assessments 
by providing greater insights into the world of college 
admissions and placement testing. Achieve analyzed 
more than 2,000 questions from college admissions and 
placement exams to determine how these tests compare 
to one another and how well they measure the college 
and work readiness benchmarks created by the American 
Diploma Project (ADP). These benchmarks are being 
used by 29 states to align high school standards, 
curriculum, assessments and accountability systems 
with the demands of college and work. 

The study was conducted with the full cooperation of 
ACT (administrator of the ACT college admissions test 
and the COMPASS placement test) and the College Board 



(administrator of the SAT and the ACCUPLACER place- 
ment test), both of whom provided Achieve with access 
to their admissions and placement exams. Achieve also 
acquired placement exams from a number of other organi- 
zations and postsecondary institutions around the country. 

Findings 

Achieve ’s ADP research shows that college faculty across 
states and institutions have a fairly consistent view of 
the rigorous level of reading, writing and mathematics 
skills that incoming freshmen need to be successful in 
first-year, credit-bearing college courses. In mathematics, 
students need knowledge and skills typically learned in a 
four-year mathematics sequence including Algebra 1 and 
II, Geometry, data analysis, and statistics. They also need 
sophisticated mathematical reasoning and problem- 
solving skills. In English, students need to be able to 
write and communicate effectively to different audi- 
ences, understand and analyze various types of complex 
informational texts, and apply sophisticated analytic and 
reasoning skills. 

Achieve’s analysis reveals that college admissions and 
placement tests vary considerably — and do not fully 
measure the knowledge and skills that are included in 
the ADP benchmarks. Generally, admissions tests were 
found to be more demanding than the placement tests 
and better balanced in the types of questions asked. 

■ Reading. College admissions tests in reading are 
more rigorous than placement tests, though the read- 
ing passages on placement tests more accurately reflect 
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the types of reading material students will encounter 
in college across the disciplines. 

■ Writing. College admissions and placement tests in 
writing are rigorous — more rigorous than most high 
school tests — and they generally reflect the kind 

of writing students will be asked to do in college. 
Institution-developed placement tests are the strongest 
of the tests analyzed by Achieve. 

■ Mathematics. Admissions and placement tests in 
mathematics emphasize algebra, which is critical for 
credit-bearing mathematics courses. However, the 
algebra content assessed tends to favor prealgebra and 
basic algebra over the advanced algebraic concepts and 
skills essential for college readiness. Although place- 
ment tests are narrowly focused on algebra, admis- 
sions tests are broader, measuring a range of other 
important topics such as data analysis, statistics and 
geometry. 

Recommendations for 
K-12 Policymakers 

To improve the preparation of high school students so 
that all graduates are prepared for college and work, 
states need a more rigorous and coherent system of assess- 
ments — one that is capable of measuring the standards 
students are expected to meet in high school and signal- 
ing readiness for postsecondary success. Very few states 
have such a system in place today. 

What are the implications of Achieve ’s study of admis- 
sions and placement exams for K— 12 leaders as they seek 
to build college-ready tests into their high schools? 

Augment admissions tests when incorporating them 
into statewide testing systems. The ACT and SAT are 

widely used by colleges and therefore have credibility 
with students, parents and the broader public. States 
that are considering incorporating these tests into their 
assessment and accountability systems, however, should 
proceed with caution. Achieve ’s analysis reveals that 
although admissions tests do some things very well, there 
are gaps in what they measure. Neither the ACT nor the 



SAT includes the full range of advanced concepts and 
skills reflected in the ADP benchmarks and, increasingly, 
in state high school standards. To be effective, states need 
to augment the ACT and SAT with additional test ques- 
tions or with additional performance measures to ensure 
stronger alignment with state standards and to assess the 
more advanced concepts and skills. Achieve encourages 
states that are considering incorporating the ACT or SAT 
into their state assessment and accountability systems to 
conduct independent alignment studies first and then 
work with ACT and the College Board to supplement the 
assessments as needed to ensure greater coherence and 
alignment. 

Consider using end-of-course tests to tap higher-level 
content and skills and place students into college 
courses. A growing number of states are pursuing end- 
of-course tests in high school as a strategy for measuring 
college readiness at the upper grades (e.g., Algebra II and 
11th grade English) while also better aligning tests with 
the high school curriculum. There are multiple benefits 
to this approach. End-of-course tests can be tied closely 
to the curriculum and to the courses that states require 
for graduation. They also are more sensitive to instruction 
because they are taken right after a student completes a 
course, and they allow states to monitor performance and 
ensure consistency of rigor across the state. For end-of- 
course tests to serve as an indicator of college readiness, 
states will need to administer tests in higher-level courses, 
such as Algebra II and 11th or 12th grade reading and 
writing. It also is essential for higher education to play 
a role in developing and/or reviewing these exams to 
ensure they reflect the skills needed for college success. If 
postsecondary institutions participate in developing these 
exams, they can more readily use them for placement 
purposes in entry-level college courses. This will send a 
powerful signal to students and their parents and teachers 
that performance in high school pays off in college. 

Modify existing high school tests to measure college 
readiness. Some states are considering adding ques- 
tions to existing high school assessments to round out 
those tests and make them adequate measures of college 
readiness. If done well, this approach has the benefit of 
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streamlining the number of tests students take by serving 
the dual purpose of measuring student mastery of con- 
tent in the state’s standards as well as indicating readiness 
for credit-bearing college courses. The tests also can alert 
students if they need additional preparation for college in 
time to adjust their senior year coursework. 

Use existing college placement tests for diagnostic 
purposes only. Some states are making college place- 
ment tests available for students to take voluntarily in 
high school. These exams can provide information to 
high school students about their readiness for credit- 
bearing, first-year college courses and allow teachers 
to work with students to address learning gaps in their 
senior year. However, placement tests should not be used 
as a substitute for building more comprehensive measures 
of college and work readiness into the state high school 
accountability system. The majority of the college place- 
ment tests reviewed by Achieve are narrowly focused on 
a subset of knowledge and skills; in math and reading 
in particular, they reflect relatively low levels of rigor. If 
states were to incorporate existing placement tests into 
their formal high school accountability systems, it might 
inadvertently lead to a narrowing and watering down of 
the curriculum. 

Recommendations for Higher 
Education Policymakers 

Developing a coherent system of high school assessments 
capable of measuring college readiness will primarily be 
driven by state K-12 education leaders, but postsecond- 
ary leaders have a critical role to play. 

Clearly define expectations for incoming students. 

Postsecondary systems and institutions must be clearer 
and more transparent about what it means to be college 
ready so that the K— 12 system has something more con- 
crete to aim for. In each state, colleges and universities 
should collaborate with the K-12 system to define and 
publicize the standards for transition from high school to 
college. These standards should be pegged to the knowl- 
edge and skills high school graduates need to succeed 
in credit-bearing, non-remedial courses. The standards 
must be tangibly articulated, much like the presentation 



of K-12 academic content standards, so that they do not 
simply become represented by a score on an admissions 
or a placement test. 

Scrutinize placement tests given to incoming students 
to determine eligibility for entry into credit-bearing 
courses. Once postsecondary institutions have clearly 
defined their expectations, it is important that they 
examine their existing placement tests (and admissions 
tests, if they are used to make placement decisions) to see 
whether they measure the content and skills needed to 
enter and succeed in credit-bearing courses — not just 
to avoid remediation. If, as Achieve found, the tests in 
use do not fully measure the rigor and breadth of college 
readiness expectations, college faculty should consider 
commissioning or developing placement tests that reflect 
the real demands of what it takes to succeed in college. 

Collaborate with K-12 on the development of high 
school tests that fully reflect the breadth and rigor 
of the content needed for success in postsecondary 
education. In addition to incorporating more challenging 
placement tests, colleges should work with K-12 systems 
to develop stronger high school tests that tap the full 
range of college-ready content — and then use the results 
of those tests to place students in courses appropriate to 
their knowledge and skills. Colleges should then align 
their qualifying scores to establish a high standard of 
preparation that applies no matter where a student decides 
to go to college. This type of coordination would have 
two distinct benefits: It would reduce confusion about just 
what is required for college-level work, and it would do so 
in a way that reduces the number of tests a student takes. 

Support the development and use of K-16 longitu- 
dinal data systems. Institutions of higher education 
need to work with K-12 systems to develop effective data 
systems that follow students from high school to col- 
lege. Without data on how students perform once they 
arrive in college, no one can be sure which high school 
programs are effective and which need improvement. An 
effective longitudinal data system should include high 
school grades and assessment results, college course- 
taking patterns, success in first-year college courses, 
and persistence and completion rates. 
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Introduction 



Study after study shows that U.S. high schools prepare only a minority of students for suc- 
cess in college. A third or more of our students fail to graduate. An alarming number of those 
who do graduate are unprepared for college-level work. Approximately 30 percent of first-year 
college students are placed immediately into a remedial college course, and the odds are high 
that these students will never complete a postsecondary degree. In response to the “broken 
promise” of a high school diploma, Achieve (in partnership with The Education Trust and 
the Thomas B. Fordham Foundation) launched the American Diploma Project (ADP). 



As its first step, ADP set out to identify the knowledge 
and skills necessary for success in college and on the 
job. Research in colleges, universities and high- 
performance workplaces across the country revealed 
that real-world expectations are significantly higher than 
what is currently expected of most high school graduates. 
ADP’s 2004 report Ready or Not: Creating a High School 
Diploma that Counts describes specific content and 
skills in English and mathematics that graduates must 
master by the time they leave high school if they expect 
to succeed in postsecondary education or in high- 
performance, high-growth jobs. These benchmarks are 
considerably more rigorous than current high school 
standards. Now, a growing number of states are working 
to close the gap between what colleges require and what 
high schools deliver. Through Achieve ’s ADP Network, 
29 states are in the process of aligning high school 
expectations with those of college and work. 

Rigorous standards are an important first step, but often 
it is the tests that dictate what gets emphasized in class- 
rooms. It is critical, therefore, that high schools measure 
whether their students have acquired the knowledge and 
skills important to college professors. Unfortunately, this 
rarely happens. In 2004, Achieve conducted a detailed 
analysis of high school graduation exams in mathematics 
and English language arts in six states (Florida, Mary- 
land, Massachusetts, New Jersey, Ohio and Texas) that 
enroll a quarter of the nation’s high school students. 1 
Achieve ’s study shows that, on average, exit exams reflect 
very modest expectations. These tests are administered 



to students in the 10th or 11th grade and tend to cover 
material that most students study in even earlier grades. 
The scores required to pass these tests can be achieved 
with the knowledge and skills expected of students in 
only 8th or 9th grade. The bottom line: These high 
school tests at best measure only a small fraction of 
the knowledge and skills that college professors say are 
needed to succeed beyond graduation. 

Many states are seeking to build more rigorous college- 
ready measures into their testing systems, and they are 
trying a variety of approaches. A growing number of 
states are incorporating national college admissions 
tests — the ACT and SAT — into their assessment and 
accountability systems. Others are funding voluntary 
early indicator assessment programs through which high 
school students take college placement tests to identify 
gaps in their preparation for first-year, entry-level courses 
in English and mathematics. States also are considering 
using existing high school assessments given late in high 
school — such as Algebra II end-of-course or 11th or 
12th grade English tests — to measure the readiness of 
high school students (tests that also might prove useful to 
colleges as placement tests) . 

What Do Transition 
Tests Measure? 

Achieve launched this study of the most frequently used 
college admissions and placement tests to help inform 
state decisionmaking regarding the inclusion of college 
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admissions and placement tests into their high school 
assessment and accountability systems. Achieve was 
interested in answering the following questions: 

■ What do the college admissions and placement tests 
measure? 

■ To what extent and how well do these tests measure 
the college- and work-ready benchmarks defined by 
Achieve ’s ADP? 

■ What are the implications of Achieves analysis of 
college admissions and placement tests for K-12 poli- 
cymakers as they design and build more robust and 
coherent systems of high school assessments? 

To answer these questions, Achieve analyzed a variety of 
admissions and placement tests encountered by students 
in the transition from high school to college. The analy- 
sis was done to determine the extent of the exams’ align- 
ment with what Achieve has learned are the knowledge 
and skills in English and mathematics that high school 
students need to acquire to meet the expectations colleges 
have for incoming students. 

Purposes of Admissions 
and Placement Tests 

College admissions and placement tests were developed 
to serve different purposes. 2 Admissions tests — the 
ACT and SAT — are intended to indicate the likelihood 
a student will succeed in a broad range of college courses. 
Most selective colleges and universities use scores from 
admissions tests in conjunction with other information, 
such as course-taking and grade point averages from 
high school transcripts, to help them determine whether 
or not to admit a student. Often, scores on admissions 
tests also are used to place students into credit-bearing 
courses, allowing them to bypass placement tests if 
their subscores in reading, writing or mathematics meet 
certain thresholds. 

In contrast, placement tests are designed specifically 
to ensure that incoming students possess the know- 
ledge and skills in reading, writing and mathematics 



necessary to gain access into entry-level, credit-bearing 
courses. Students who fare poorly on placement tests are 
assumed to have gaps in their preparation and are advised 

— and often required — to take one or more remedial 
courses to learn (or re-learn) important material prereq- 
uisite for credit-bearing courses. (In some cases, and 
especially in mathematics, additional rounds of place- 
ment testing are necessary to make a more refined 
placement decision, i.e., which level of remedial or 
credit-bearing course is most appropriate for the student.) 
Students who pass the placement test (or are exempted 
from the placement requirement) are granted access to 
the first college-level course in English and mathematics, 
as well as to all credit-bearing, entry-level courses offered 

at the institution. In this way, placement tests — or a set 
of placement tests — serve both the narrow function of 
identifying specific deficiencies in student preparation 
and the broad function of certifying that the student is 
ready for all college -level work. 

The Placement Process 

Typically colleges offer students options for demonstrat- 
ing the required competencies: achieving a satisfac- 
tory score on the SAT or ACT or on a placement exam 

— the most common of which are ACCUPLACER and 
COMPASS. Qualifying scores are set by each institution, 
often under the influence of guidelines provided by state 
agencies or test publishers. Placement procedures — and 
qualifying scores — at different colleges and universities 
are idiosyncratic, varying from campus to campus. Many 
involve multiple tests or sequenced decisions. 

Recognizing that many students enter college unprepared 
for the first credit-bearing college writing or mathematics 
course and for college-level work in general, most colleges 
and universities offer several options to help students 
close their preparation gaps. Some of these options are 
strictly remedial in nature: They focus almost entirely on 
knowledge and skills that most students learn in grades 
6-12. These preparatory courses, often termed “develop- 
mental,” typically do not offer credit that counts toward 
a college degree, although they may count for credit for 
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The American Diploma Project Network 



Research conducted by the American Diploma Project 
(ADP) in colleges, universities and high-performance 
workplaces across the country shows that real-world 
expectations are significantly more rigorous than 
current high school standards. Indeed, there is an 
“expectations gap” between what colleges expect of 
incoming freshmen and what high school graduates 
actually know and can do. 

ADP’s 2004 report Ready or Not: Creating a High 
School Diploma that Counts describes specific content 
and skills in English and mathematics that graduates 
must master by the time they leave high school to 
succeed in postsecondary education or in high- 
performance, high-growth jobs. The ADP bench- 
marks are being used by a number of states to align high 
school expectations with those of college and work and 
close the expectations gap. Twenty-nine states — col- 
lectively responsible for the education of more than half of 
the students in the United States — have formed the ADP 
Network and are committed to four specific actions: 

Align high school standards and assessments with 
the knowledge and skills required for success after 
high school. Too often, state high school standards are 
not anchored in the skills and knowledge employers and 
colleges now demand. As a result, high school graduates 
can believe they have done well but still be unprepared 
for productive work and further learning. 

Require all high school graduates to take challeng- 
ing courses that actually prepare them for life after 
high school. New research by ADP and ACT indicates 
that whether students go directly to college or into the 
workforce after graduation, they need the same level of 
knowledge and skills, particularly four years of rigorous 
English and mathematics. Other research suggests that 
more rigorous study reduces the gap in college comple- 
tion rates between African American and Latino students 
and their white peers. Equally important, students who 
take more rigorous courses are significantly more likely to 
succeed in postsecondary education and the workplace. 




□ ADP Network State 



Build college- and work-ready measures into state- 
wide high school accountability systems. According 
to Achieve ’s research, very few states currently measure 
students’ readiness for college and work. High school 
exit exams measure content that students typically learn 
in early high school, which is only a subset of knowledge 
and skills they will need after graduation. To help ensure 
that students graduate prepared for college and work, 
states will need to build more rigorous measures into 
their accountability systems. 

Hold high schools accountable for graduating stu- 
dents who are ready for college or careers, and hold 
postsecondary institutions accountable for students’ 
success once enrolled. The mission of high schools 
is to prepare all students for success in college, careers 
and citizenship. Yet in most states, high schools rarely 
are held accountable for ensuring that students achieve 
these goals. Colleges, meanwhile, have few incentives 
to improve retention and completion rates. With the 
number of jobs requiring at least some education beyond 
high school expanding rapidly, states must work closely 
with higher education systems to ensure that students 
who begin college actually graduate with a degree and a 
productive set of skills and knowledge. 
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purposes of full-time status and financial aid. Increas- 
ingly, four-year colleges and universities are neither 
willing nor permitted to teach developmental courses to 
their students. Instead, students often are sent to com- 
munity colleges or other sub-baccalaureate institutions 
for remediation. 

Other course options are designed to provide the added 
support required to help students succeed without the 
cost, lost time or stigma of repetition and remediation. 
These courses are typically versions of Comp 101 or Col- 
lege Algebra expanded in either duration or intensity to 
provide additional, targeted instruction. 

In mathematics, placement testing and decisionmaking 
have several additional complications. Across the spec- 
trum of “entry-level” courses offered at most institutions 
of higher education are five different mathematics courses 
that form a stairway from elementary to advanced. Two 
of these five — Arithmetic and Elementary Algebra (i.e., 
high school Algebra I) — are universally considered 
remedial. Often, but not always, the third (Intermedi- 
ate Algebra) is as well. Intermediate Algebra is similar to 
high school Algebra II; so as more and more states 
require this course to be taken by all high school stu- 
dents, colleges may increasingly insist that it is remedial. 



MATHEMATICS COURSE TITLES 



High School 


College 


Algebra 1 = 


Elementary Algebra 


Algebra II = 


Intermediate Algebra 


Precalculus = 


College Algebra 


Calculus = 


Calculus 



Trigonometry is sometimes included in Precalculus 
or College Algebra, but it also is often an optional 
supplement. 



The most common introductory college mathematics 
course taken at two- and four-year colleges is College 
Algebra, 3 sometimes called Precalculus. This course 
covers advanced topics that form the transition between 
Algebra II and Calculus. Some version of this course 
is taught in almost every high school; recent variations 
include versions that emphasize mathematical modeling. 
Accelerated high school students take Precalculus prior to 
Advanced Placement Calculus while other college-bound 
students take it in their senior year. Except at the most 
selective institutions, College Algebra is widely accepted 
as a transfer-worthy, credit-bearing college mathematics 
course. The syllabus of a typical College Algebra course 
covers topics that are prerequisite to Calculus, the fifth 
and top step in the ladder of introductory mathemat- 
ics courses. It is worth noting that different major areas 
of study at higher education institutions have different 
prerequisites in mathematics, further complicating this 
picture. For students intending to major in mathemat- 
ics or one of the many “mathematics-intensive” majors 

— such as business, engineering or the natural sciences 

— Calculus may be the first entry-level course that 
counts for mathematics credit. 

Placing students satisfactorily into this broad range of 
mathematics courses requires a fairly complex system of 
placement exams — a system that typically involves more 
than one level of placement exam. Many forces influence 
the way placement tests are used, including admissions 
policies, political decisions and enrollment management 
(both for remedial and credit-bearing courses). A key 
factor is the establishment of “cut scores” that define the 
threshold for placement into one or more courses. Cut 
scores used for similar placement purposes often vary from 
institution to institution, representing different levels of 
preparation — even for courses with the same name. 
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Achieve's Study 

Overview of Methodology 

The method of analysis used for this study was devel- 
oped for Achieve’s earlier study of high school exit exams 
and adapted for use with admissions and placement 
tests, with the assistance of content experts from Michi- 
gan State University. ACT and the College Board also 
provided helpful feedback as protocols were revised. Two 
or more trained experts analyzed each test, coding each 
item on a number of dimensions of test quality. Experts 
worked independently and reconciled any differences in 
their judgments, resulting in a consensus on each item. 
These detailed analyses then were aggregated to build an 
overall picture of each test, which enabled comparisons. 

The first dimension examined was the content measured 
by each test. In particular, Achieve wanted to know 
what content was being measured by each item and how 
advanced that content was. To determine this in math- 
ematics, Achieve used an international scale created as 
part of the Third International Mathematics and Science 
Study (TIMSS). The framework provides a detailed, com- 
prehensive taxonomy of mathematics content organized 
at its most general levels according to the following major 
domains of mathematics: number, algebra, geometry/ 
measurement and data analysis/statistics. These broad 
categories then are broken down into smaller units to 
allow for finer-grained comparisons. For example, geom- 
etry content is divided into a variety of categories such as 
two-dimensional geometry and measurement; three- 
dimensional geometry and measurement; transformations, 
congruence and similarity; and trigonometry. In English 
language arts, Achieve developed a scale to categorize 
the content of each item, such as vocabulary or literary 
elements. 

The second important dimension considered was the 
complexity (or cognitive demand) of each item — what 
the item asks students to do with their knowledge. In 
reading, for example, students can be asked to simply 
recall information from a text — a relatively low-level 
skill — or they might be expected to perform a more 



complex task such as comparing an idea across two pas- 
sages. In mathematics, they might be asked to perform a 
simple computation or procedural process, or they can be 
challenged to reason though a multistep problem. 

A third dimension, relevant only to the reading tests, is 
the complexity of the reading passages. This dimension 
figured prominently into the analysis because the interac- 
tion of the cognitive demand of an item and the difficulty 
of a passage contributes to the overall rigor of the test. 

To address this dynamic, Achieve developed a Reading 
Rigor Index (RRI). In addition, the genre of reading 
passages was considered using the National Assessment of 
Educational Progress (NAEP) 2009 Reading Framework, 
which recommends two types of texts be included on the 
assessment: literary texts, which include fiction, literary 
non-fiction and poetry; and informational texts, which 
include exposition, argumentation and persuasive text, 
and procedural text and documents. 

A fourth dimension, relevant only to the multiple-choice 
writing items, is the level of challenge of each item, 
which captures the interplay between the specific skill 
targeted by the item and the context in which it is used. 
This lens was adapted from the ACT Standards for 
Transition for Writing and includes skills in a number 
of categories (e.g., topic development, organization and 
word choice) at varying levels of complexity. Direct writ- 
ing assessments were examined more holistically, with 
attention to those elements most necessary for success in 
the college classroom as delineated in the ADP English 
benchmarks. 

To assess the difficulty of mathematics items, Achieve 
used the International Grade Placement (IGP) index, a 
composite among the 40 TIMSS countries (other than 
the United States) that shows when the curriculum 
focuses on different mathematics content — at which 
point the highest concentration of instruction on a topic 
occurs. The average international grade level for each test 
then is calculated by averaging the IGP indices of each 
item. 

Finally, Achieve determined the ways in which the tests 
reflect the content and skills in the ADP benchmarks, 
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which college faculty from across the disciplines agree are 
necessary for success in postsecondary pursuits. In math- 
ematics, these benchmarks represent content found in a 
rigorous four-year course sequence that includes content 
found in courses like Algebra I and II and Geometry, as 
well as considerable data analysis and statistics. The Eng- 
lish benchmarks demand strong oral and written com- 
munication skills and considerable research and analysis. 
Logic and reasoning skills also are a critical element of 
the benchmarks. To identify the English and mathemat- 
ics knowledge and skills that high school graduates need 
to be prepared for success in college and work, Achieve 
worked with more than 300 faculty members from two- 
and four-year institutions from a variety of academic 
disciplines (as well as front-line managers from high- 
skilled jobs). 

For a detailed description of Achieve ’s methodology in 
reading, writing and mathematics, see Appendices B-D. 

The Tests Analyzed by Achieve 

Admissions and placement exams in English (reading and 
writing) and mathematics can be grouped broadly into 
four categories, each of which was included in this study: 

■ National Admissions Tests. Each year, more than 
2 million college applicants in the United States 
take one of two national admissions tests — the 
SAT (developed by the College Board) or the ACT 
(developed by ACT). Colleges and universities use 
these admissions tests as one factor in admissions 
decisions. Many colleges and universities also permit 
results on the SAT and ACT to exempt students from 
taking subsequent placement tests or to substitute 
for a placement test score. According to the College 
Board, the SAT is intended to measure critical 
thinking skills required for academic success in 
college. In contrast, the ACT is explicitly curriculum 
based, designed to assess what students have learned 
in their high school courses. 

■ National Placement Tests. Colleges and universities 
use placement tests to help determine the appropriate 
course level for students. Two nationally distributed 
tests — ACCUPLACER (developed by the College 



Board) and COMPASS (developed by ACT) — 
are widely used for placement purposes, especially 
by community colleges, which often use them to 
place students in developmental courses. Both the 
ACCUPLACER and the COMPASS are computer- 
adaptive tests, meaning they are able to adapt to a 
test taker’s ability level by successively selecting 
questions from a large pool of items based on 
whether the previous question was answered correctly. 
Typically, a computer-adaptive test begins with a 
question of moderate difficulty. If answered correctly, 
it is followed by a more difficult question; if answered 
incorrectly, it is followed by an easier question. In 
this way, it appears to the test taker that the difficulty 
of the exam is tailored to his or her level of ability. 
Compared to a non-adaptive test with a fixed set of 
items, computer-adaptive tests require fewer test 
items to arrive at a final score, which in turn requires 
less time. 4 

For this study, Achieve examined a sample of the test 
questions from the total item pool of these computer- 
adaptive tests, focusing in particular on the tests 
that are most commonly used by institutions for 
placement into entry-level, credit-bearing courses 
in English (e.g., Comp 101) and mathematics (e.g., 
College Algebra). The College Board helped identify 
the ACCUPLACER tests most commonly used 
for placement into such courses: in English, the 
ACCUPLACER Reading Composition and Sentence 
Skills tests; in mathematics, the ACCUPLACER 
Elementary Algebra. To confirm this — and to 
identify which ACT tests are most commonly used 
for placement into entry-level, credit-bearing courses 
— Achieve surveyed postsecondary institutions from 
a number of states in the ADP Network. The results 
of Achieve ’s survey indicate that the ACT computer- 
adaptive placement tests most commonly used for 
placement into entry-level, credit-bearing courses are 
the COMPASS Reading, Writing and Algebra tests. 
Achieve ’s survey found that the ACCUPLACER 
College Level Math test and the COMPASS College 
Algebra test also are used for this purpose — and they 
are used for placement into Calculus. For this reason, 
those tests are included in the group of advanced-level 
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tests. Although both ACT and the College Board 
offer additional tests (including ACCUPLACER’s 
Arithmetic and COMPASS’ Numerical Skills, 
Geometry and Trigonometry), Achieve chose to 
analyze only those tests that are most commonly used 
for placement into entry-level, credit-bearing courses. 
Thus, there are items from both vendors’ item pools 
that are not captured in this study. 

■ State or Systemwide Tests. Some states (e.g., Texas) 
and higher education systems (e.g., California State 
University and Washington state) have developed 
placement tests for common use across institutions. 
Generally, these tests are designed to match and sup- 
port articulation agreements for credit transfer among 
institutions of higher education. 

■ Institution-Level Tests. Many colleges prepare their 
own placement tests and procedures to fit their unique 



curricular needs. Several institution-level mathemat- 
ics placement tests are (with permission) modeled on 
or are minor variations of the Mathematical Associa- 
tion of America (MAA) placement tests. The MAA 
(in conjunction with MapleSoft) is now adapting its 
placement tests to a computer-based environment. 

For this study, Achieve analyzed examples of each type 
of test, which were generously provided by the compa- 
nies and institutions that volunteered to participate. The 
tables below and on page 12 list all of the tests included 
in the study; Appendix A describes the tests in greater 
detail. For mathematics, separate placement tests often 
are developed to assess readiness for College Algebra 
and Calculus. Although the bulk of this report focuses 
on tests that assess readiness for College Algebra — 
typically the first credit-bearing course in college 
mathematics — both levels of the tests were examined 
and are included in the table below. 



TESTS INCLUDED IN STUDY: MATHEMATICS 



Test Category 


Mathematics: College Algebra 


Mathematics: Calculus 


National 

Admissions 


ACT Assessment — Math (two forms) 






Tests 


■ SAT Reasoning Test — Math (two forms) 






National 


ACCUPLACER — Elementary Algebra 




ACCUPLACER — College Level 


Placement 






Mathematics 


Tests 


■ COMPASS — Algebra 


■ 


COMPASS — College Algebra 


State or 


Texas Higher Education Assessment 


■ 


Washington Advanced Mathematics 


Systemwide 

Tests 


Washington Intermediate Mathematics 
Placement Test 




Placement Test 


Institution- 


Mathematical Association of America 


■ 


Louisiana State University College 


Level Tests 


Algebra Placement Test 




Algebra Placement/Credit Exam 




Purdue University-North Central 


■ 


Mathematical Association of America 




Student Assessment and Measurement 




Calculus Readiness Placement Test 




Placement Test 




University of Maryland-College Park 




Temple University Mathematics 




Math Placement Test 




Placement Test 




University of Minnesota Calculus 




University of Minnesota College Math 




Readiness Test 




Readiness Test 
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TESTS INCLUDED IN STUDY: ENGLISH LANGUAGE ARTS 



Test Category 


Reading 


Direct Writing 


Indirect Writing 


National 


ACT Assessment — 


ACT Assessment — 


ACT Assessment — 


Admissions 


Reading (two forms) 


Writing 


English (two forms) 


Tests 


SAT Reasoning Test — 


SAT Reasoning Test — 


SAT Reasoning Test — 




Critical Reading 
(two forms) 


Writing 


Writing (two forms) 


National 


ACCUPLACER — 


ACCUPLACER 


ACCUPLACER — 


Placement 


Reading 


WritePlacer 


Sentence Skills 


Tests 


Comprehension 


COMPASS e-Write 


■ COMPASS — 




COMPASS — Reading 




Writing Skills 



State or Texas Higher Education California State Texas Higher Education 

Systemwide Assessment University Early Assessment 

Tests Assessment Program — 

Direct Writing 

Texas Higher Education 
Assessment 



Institution- ■ The College of 

Level Tests New Jersey Writing 

Exemption and 
Placement Test 

■ Louisiana State 

University Writing Test 

Purdue University- 
North Central English 
Composition Placement 
Essay 

Temple University 
English Placement 
Test — Essay 
Component 

University of 
Maryland-Baltimore 
County English 
Composition Placement 
Essay 

University of 
Massachusetts- 
Amherst Writing 
Placement Exam 

Youngstown State 
University Composition 
Placement Test 
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Findings: Reading 



SUMMARY 

To be ready for college, students need to be able to read and interpret a variety of complex 
texts, both fiction and non-fiction. In fact, the majority of what students read in college is 
informational in nature, such as textbooks and articles from periodicals, which may contain 
technical information, charts and graphs. Admissions and placement tests should reflect the 
full range of diverse and challenging reading students are likely to do in college. 

Achieve's analysis of admissions and placement tests revealed that: 

■ Reading passages on admissions tests are complex and reflect the demands of college and 
work, but placement tests include less challenging passages that are more in line with the 
level of reading done in middle school and early high school. 

■ Placement tests emphasize informational text — the types of reading students tend to do 
in college and on the job. Admissions tests are more balanced between informational and 
literary texts. 

■ Overall, admissions tests present a far greater level of rigor than the placement tests. 



Because reading is the focus of early education, it is 
often somewhat taken for granted in high school and 
college curricula. College educators, however, expect a 
variety of sophisticated reading skills that develop only 
through extensive and disciplined study of varied texts. 
ADP embeds many of these reading skills in its English 
benchmarks. A few of many examples include the need 
for high school graduates to be able to identify interrela- 
tionships between and among ideas and concepts within 
a text; analyze the ways in which a text’s organizational 
structure supports its meaning; evaluate the quality of 
evidence used to support an argument; and interpret 
information presented in maps, tables and diagrams . 5 

Because the ability to read and to learn through reading 
is so central to college education, many colleges require 
entering students to demonstrate a minimum level 
of competence in reading. In some institutions this 
competence is assessed during the admissions process, 
most commonly by scores on the SAT or ACT tests. 



Because factors other than test scores contribute to admis- 
sions decisions, many admitted students (especially those 
for whom English is not their native language) may not 
possess the reading skills required for college work. Read- 
ing placement tests provide a second opportunity for 
colleges to identify students who may need special help. 

With rare exceptions, colleges do not give academic 
credit for courses in reading. The goal of a reading place- 
ment test is solely to identify students whose reading 
skills are so weak that they are not likely to do well in 
any credit-bearing course. Typically, students who score 
poorly on a reading placement test are advised (and 
sometimes required) to take special refresher courses 
during the summer between high school and college or 
various non-credit remedial reading courses during their 
first semester of college. Most postsecondary institutions 
limit either the number or kind of regular credit courses 
that students who require remedial reading may take 
until this deficiency is overcome. 
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For this study, Achieve analyzed reading tests that are 
used for placement purposes by colleges and universities, 
including: ACT; SAT; sets of items that sampled both the 
ACCUPLACER and COMPASS (placement tests that 
are used by numerous institutions across the country); 
and the Texas ffigher Education Assessment, which is a 
state-level placement instrument. 

In total, the reading tests analyzed for this study include 
more than 500 items. Achieve ’s experts coded items 
on each of these tests in several categories: the overall 
difficulty of the reading passages, the genre of the read- 
ing passages, and the content and cognitive demand of 
the questions. Additionally, the cognitive and reading 
passage demand scores were combined into an index 
that measures overall rigor. Reading passages on the 
ACCUPLACER average only a few sentences in length; 
therefore, they were not given difficulty ratings (or a 
rigor score) because the scale is designed to measure 
longer passages. 

The result of this extensive analysis is a rich database 
of information with which to evaluate the tests and 
compare their characteristics. The following sections 
describe broad patterns that emerged from this analysis. 
The Texas assessment — a particularly strong test — is 
the only statewide placement test in the study; thus, it is 
highlighted in a sidebar (see page 18). 



TESTS ANALYZED IN READING 



Test Category 


Reading 


National 


ACT Assessment — 


Admissions 


Reading (two forms) 


Tests 


SAT Reasoning Test — 
Critical Reading 
(two forms) 


National 


ACCUPLACER — Reading 


Placement 


Comprehension 


Tests 


COMPASS — Reading 



State or Texas Higher Education 

Systemwide Assessment 

Tests 



How challenging are the reading 
passages? 

College students will read a rich array of complex texts. 
To judge the complexity of reading passages on admis- 
sions and placement tests, Achieve ’s reading experts cre- 
ated a six-point scale describing texts from the relatively 
simple to the very complex. Simple texts (Level 1) tend 
to be concrete, use a straightforward organizational 
structure and basic vocabulary, and most closely resemble 
an upper-elementary school reading level. At the high- 
est level of complexity (Level 6), texts are more abstract. 
They can include specialized vocabulary and complex 
organization and syntax. Level 5 and 6 texts represent 
late high school reading. Students who can read and 
comprehend texts at these top levels have a solid founda- 
tion and likely are prepared for college-level reading. 

Admissions Tests 

Achieve ’s analysis showed that the ACT and SAT on 
average target two-thirds of their items at the upper-level 
passages (Levels 5 and 6), which — as late high school 
reading — are sophisticated and complex reading. 

Placement Tests 

In contrast, less than one-quarter of the COMPASS 
reading items analyzed in Achieve ’s study are directed 
at upper-level passages, and nearly half are Level 3 — 
late middle school. (The ACCUPLACER items were 
not included in these data because the test presents 
shorter texts that were not given a reading level on the 
Achieve scale.) 

Do the passages reflect the types of texts 
students will read in college? 

In addition to complex reading passages, a true test of 
college readiness ought to include a variety of genres that 
reflect the full range of reading required throughout the 
college curriculum, including informational texts (e.g., 
expository and persuasive/argumentative) similar to what 
students will encounter in business, science and history 
courses, and literary texts (e.g., fiction/poetry and liter- 
ary non-fiction). That said, most of what students read 
in college is informational in nature, such as textbooks. 



14 



Achieve, Inc. 





primary source documents and articles 
from periodicals, which may contain 
technical information, charts and 
graphs. 

Admissions Tests 
Both the ACT and SAT emphasize 
literary texts, with more than 60 
percent of their items attributed to 
literary passages. In this regard, they 
resemble the high school exit exams 
analyzed by Achieve in its earlier study, 
which reflected an emphasis on literary 
study that remains the focus of most 
high school English courses. 6 College 
students likely will need to read and 
interpret the quality of information 
and evidence presented in a wide range 
of materials. The reading passages on 
the ACT and SAT currently do not 
reflect the full range of these materials. 
Neither, in large part, do high school 
English courses, which continue to 
emphasize literature at the expense of 
informational text. 

Placement Tests 

In contrast, Achieve ’s analysis indicates 
that ACCUPLACER and COMPASS 
both emphasize informational texts, 
with more than three-quarters of the 
sample of items targeting the kinds 
of reading students will encounter 
in classes other than literature. 7 This 
balance is in line with the frameworks 
for the 2009 National Assessment of 
Educational Progress (NAEP) 12th 
Grade Reading Test, 8 which have 
recently been revised to better reflect 
readiness for college, work and the 
military. These new frameworks 
designate that the distribution of 
NAEP passages will be 70 percent 



Complexity of Passages on Reading Tests 



100% 




90% 


— 


80% 


— 


70% 


— 


60% 


— 


50% 


— 


40% 


— 


30% 


— 


20% 


— 


10% 


— 


0% 





17 % 



ACT SAT 

Admissions 



i I Level 3: Late Middle School 
PI Level 4: Early High School 
I Levels 5 and 6: Late High School 




COMPASS 



Placement 



Note: The ACCUPLACER items were not included in these data because the test 
presents only sentences or short paragraphs that were not given a reading level 
on the Achieve scale. 

Totals may not equal 100 percent due to rounding. 



Content on Reading Tests 



100% 
90% 
| 80% 
£ 70% 
o 60% 
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| 40% 
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20 % 

10 % 

0 % 




ACT SAT ACCUPLACER COMPASS 



Admissions 



Placement 



■ Literary Passages H Informational Passages 



Totals may not equal 100 percent due to rounding. 
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Cognitive Demand of Reading Tests 




Admissions Placement 

I ! Level 1: Literal Recall H Level 3: High Inference 

iff! Level 2: Low Inference H Level 4: Analysis 



Totals may not equal 100 percent due to rounding. 



informational and 30 percent literary, 
a balance that matches the relative 
emphasis of the postsecondary world. 

How challenging are the 
reading questions? 

In addition to its reading passages, 
the character of a reading test depends 
on the level of cognitive challenge of 
the questions asked about those pas- 
sages. To analyze cognitive demand of 
the questions on the admissions and 
placement tests, Achieve used a four- 
point scale with Level 1 (literal recall 
of information) being the lowest and 
Level 4 (analysis of ideas) the highest. 9 
For example, a Level 1 item might ask 
students to recall specific facts or figures 
from the passage and choose among 
them in the answer choices. A Level 4 
question might ask students to speculate 
about what assumptions influenced the 
author’s central argument. According to ADP research, 
the ability to analyze and develop ideas and to under- 
stand persuasive arguments — assessed by Level 4 ques- 
tions — is critically important for college students. 

Admissions Tests 

Achieve found that 40 percent of the questions on the 
admissions tests measure higher-level inference skills 
(Level 3 on the four-point scale), such as making 
complex inferences, making comparisons or providing 
examples. Very few questions ask students to analyze 
texts (Level 4), while more than 50 percent of the items 
on admissions tests focus on what Achieve considers to 
be basic reading skills: literal recall and low inference. 
Literal-recall questions ask students to locate explicit 
information in the text; low-inference questions require a 
slight interpretation or a straightforward generalization. 



Placement Tests 

Achieve found that the ACCUPLACER and 
COMPASS tests vary in their cognitive demand. The 
ACCUPLACER has a high proportion of high-inference 
questions — nearly half — and very few literal-recall 
questions. In contrast, more than 70 percent of the 
analyzed COMPASS items address basic literal-recall 
and low-inference reading skills. Very few items on 
either test require analysis, which is regarded as the most 
demanding performance and is cited as an important 
skill by college professors. 

Overall, how rigorous are the reading 
tests? 

It is likely that college students will be asked to perform 
high-level cognitive tasks — such as analyzing and 
interpreting — in association with reading complex texts. 
Thus, tests that pose cognitively challenging questions 
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Rigor Levels of Reading Tests 




Admissions 



I I Low Rigor 
Hi Moderate Rigor 
H High Rigor 



Totals may not equal 100 percent due to rounding. 



about rigorous reading passages are 
useful in determining whether a 
student is ready for the robust reading 
she or he will likely do in college 
courses. To capture this important 
interplay between the question and the 
passage, Achieve developed a Reading 
Rigor Index (RRI), a continuum 
on which the lower levels represent 
less demanding questions about 
less complex reading passages and 
the higher levels reflect challenging 
questions about complex texts. 10 

Admissions Tests 
Generally, the ACT and SAT present a 
moderate RRI, with the bulk of their 
items falling in the middle of the con- 
tinuum. This is a result of the fact that 
these tests tend to include relatively 
complex reading passages even though 
the passages may be coupled with ques- 
tions that are less cognitively challeng- 
ing. On average, only 7 percent of the items on the ACT 
and SAT fall at the upper end of the RRI continuum, 
where the questions and the reading passages are the 
most complex; an equal proportion falls at the lower end, 
reflecting less demanding questions about less complex 
reading passages. 



Placement Tests 

Achieve ’s analysis found that nearly half of the analyzed 
COMPASS questions fall at the lower third of the RRI 
— an indication that the questions tend to combine low 
cognitive challenge with low-level texts. (As stated earlier, 
the ACCUPLACER was not given an RRI score due to 
the brevity of its passages.) 
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The Texas Higher Education Assessment 



Since 1989, Texas colleges and universities have 
administered the Texas Higher Education Assess- 
ment in reading, writing and mathematics to assess 
the skills of entering freshman-level students. 

The test is notable for the higher-level cognitive 
demand of its items in both reading and indirect 
writing. 

Reading 

Like the national placement tests, the Texas assess- 
ment includes a large proportion of informational 
texts. Yet the test is unique in the range, variety 
and authenticity of its informational passages. 

Each passage is an example of the kind of reading 
students are likely to encounter in college. From a 
section in a sociology textbook to a student letter 
in the college newspaper, the reading passages 
acknowledge the realities of college-level demands. 



Cognitive Demand of Reading Tests 



100 % 
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« 80% 
£ 70% 
o 60% 
| 50% 
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1% 



37% 



49% 



13 % 



Placement 



17 % 

36% 



38% 



10 % 



Texas 



I I Level 1: Literal Recall U Level 3: High Inference 

IB! Level 2: Low Inference H Level 4: Analysis 

Totals may not equal 100 percent due to rounding. 

Cognitive Demand of Indirect Writing Tests 



The reading levels of the test passages are not 
as demanding as the national admissions tests 
reviewed by Achieve; however, the Texas assess- 
ment includes a high proportion of questions (17 
percent) that require analysis of those texts. As a 
result, the average level of cognitive demand of 
the Texas test is higher than that of any other test 
in Achieves study. Typically, analysis questions 
require the test taker to analyze the basis of an 
argument or a position implied in a reading selec- 
tion, going beyond merely identifying the author’s 
argument. This type of “deep reading” is vital to 
understanding and appreciating the myriad view- 
points encountered in college courses. 

indirect Writing 

In contrast to the national admissions and place- 
ment tests, which place their heaviest emphasis on items 
at the lower levels of the cognitive demand scale, the 
majority of items on the Texas assessment (58 percent) 
call for analysis. In general, these analysis items require 
the test taker to analyze an entire passage to determine 
its tone and purpose and choose the correct answer. This 




Admissions Placement Texas 



H Level 1: Literal Recall |HI Level 3: High Inference 

HI Level 2: Low Inference H Level 4: Analysis 

Totals may not equal 100 percent due to rounding. 

kind of consideration of purpose and audience is a task 
that every writer must grapple with, and answering this 
kind of question involves making a critical judgment. The 
Texas assessment is unusual in the set of tests examined in 
Achieve ’s study because it includes such a high percentage 
of critical judgment items, although a few of these kinds 
of items also were found in the admissions tests. 



18 



Achieve, Inc. 















Findings: Writing 



SUMMARY 

To be prepared for college, students need to be strong writers, capable of using evidence and 
clear reasoning to stake out a position and persuade their readers. Students also need a solid 
foundation in writing fundamentals, such as grammar and punctuation, word choice, sen- 
tence structure, organization, and style. 

Achieve's analysis of admissions and placement tests in which students were asked to produce 
a sample of writing revealed that: 

■ Admissions and placement tests require students to take a position or develop a point of 
view — important skills valued by college professors. 

■ Institution-based placement tests are particularly strong — and reflect "real-world" writ- 
ing situations by requiring students to use provided reading passages as evidence. 

■ Most scoring rubrics and anchor papers examined reflect the college-ready writing expec- 
tations of institutions of higher education. 

Achieve's analysis of admissions and placement tests in which students answered multiple- 
choice items regarding editing and revising revealed that: 

■ In general, these tests — while useful measures of students' facility with editing and 
revising — tend to focus on less challenging editing tasks rather than on more advanced 
issues of organization or development of ideas. 



In nearly all colleges and universities, the first credit- 
bearing course in English is a required course devoted 
primarily to composition. 11 The goal of this course is to 
introduce students to college-level writing, a fundamen- 
tal skill that will be used without much further instruc- 
tion throughout students’ college careers. Some students 
place out of this requirement based on an Advanced 
Placement course taken in high school; many more need 
extra instruction to fully prepare for the demands of 
Comp 101. Colleges typically use a placement test to 
advise new students about their preparation for Comp 
101. Permission to take Comp 101 also allows students 
to enroll in other entry-level courses, some of which may 
have heavy writing requirements. For all practical 



purposes, to be ready for Comp 101 is to be a college- 
ready writer, although Comp 101 also is intended to 
further hone students’ writing skills. 

Writing tests measure students’ writing ability in two 
very different ways: directly by asking students to write 
a short passage and indirectly by using multiple-choice 
questions to evaluate students’ knowledge of grammar, 
organization and style. Although direct assessments more 
accurately reflect students’ ability to compose, indirect 
assessments are more widely used because they rely exclu- 
sively on multiple-choice questions and are much easier, 
faster and cheaper to grade. Indirect assessments often 
focus on elemental aspects of writing, such as grammar, 
usage, sentence structure and punctuation. Rarely do 
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items focus on more subtle aspects, such as strategies, 
organization, style and word choice. Achieve analyzed 
the direct and indirect writing assessments separately. 

Direct Writing Assessments 

For a direct writing test to reveal whether students pos- 
sess the ability to write at the college-ready level, it must 
challenge students to produce the type of writing called 
for in college courses. Much of college writing demands 
that students analyze evidence and persuasively argue 
in favor of a position based on that evidence. Scor- 
ing rubrics and sample papers set the bar for whether a 
student’s writing is at the college-ready level. Achieve ’s 
writing experts examined a number of direct writing 
assessments, including national admissions tests (SAT 
and ACT), national placement tests (ACCUPLACER 
WritePlacer and COMPASS e-Write), and a sample of 
placement tests developed at the state and institution 
levels. 

Do writing tests reflect the type(s) of 
writing college students must be able to 
produce? 

College students are asked to write about a multitude of 
topics in a variety of subject areas, including literature, 
history, science, economics and even mathematics. Some 
assignments may call for students to summarize informa- 
tion from their assigned readings or describe a lab experi- 
ment, others may ask students to compare and contrast 
opposing positions, and still others may require analysis 
of literary themes. What professors expect in almost all 
college writing, regardless of the disciplines, is that stu- 
dents will need to establish a thesis and argue a position 
using evidence to support it in an attempt to persuade 
the reader. These skills are valued by college professors 
and are emphasized in the ADP benchmarks. 

Admissions and Placement Tests 
The majority of the admissions and placement writing 
tests examined by Achieve require students to take a stand 
and back it up with clear reasoning and convincing evi- 
dence, something they will be called upon to do through- 
out their college education and into their careers. In this 



TESTS ANALYZED IN DIRECT WRITING 



Test Category 


Direct Writing 


National 


ACT Assessment — Writing 


Admissions 




Tests 


SAT Reasoning Test — 




Writing 


National 


ACCUPLACER WritePlacer 


Placement 




Tests 


COMPASS e-Write 



State or California State University 

Systemwide Early Assessment Program — 

Tests Direct Writing 

Texas Higher Education 
Assessment 



Institution- 
Level Tests 



The College of New Jersey 
Writing Exemption and 
Placement Test 

Louisiana State University 
Writing Test 

Purdue University-North 
Central English Composition 
Placement Essay 

Temple University English 
Placement Test — Essay 
Component 

University of Maryland- 
Baltimore County English 
Composition Placement Essay 

University of Massachusetts- 
Amherst Writing Placement 
Exam 

Youngstown State University 
Composition Placement Test 



regard, the tests are different than what most high school 
students are used to seeing. Although writing assessments 
at the high school level sometimes require students to take 
a position (e.g., a prompt from a state writing test asks 
students to write an essay convincing the school principal 
to accept his or her point of view on how watching televi- 
sion affects grades), they just as often ask students to write 
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a personal narrative or create a fictional story. Although 
narrative writing is certainly appropriate for students to 
engage in during the course of their education, it gener- 
ally is not the kind of writing college students will employ 
in the majority of their coursework. 

Do college admissions and placement 
writing tests ask students to produce 
written text in the same way they will be 
asked to in college? 

In general, college students write about the content they 
are learning. To do so, they likely are called upon to 
comprehend and engage with ideas conveyed through 
written text. 12 Thus, Achieve sought to understand 
the extent to which admissions and placement tests 
approximate the actual writing situations students will 
face in college. Achieve reviewed the writing prompts 
and supporting materials to determine the extent to 
which students are called upon to take a position based 
on evidence and what the nature of that evidence tends 
to be. The findings show that although the admissions 
and placement tests often require students to produce 
similar types of writing, they take somewhat different 
approaches. 

National Admissions and Placement Tests 
National admissions and placement tests (ACT, SAT, 
WritePlacer and e-Write) tend to provide a short quota- 
tion or synopsis of conflicting positions and ask students 
to use prior knowledge or experience as evidence in 
support of the position they decide to take. For the sake 
of fairness, these “prompts” must be easily understood 
and generic enough for students to relate to the topic and 
have a reasonable chance to produce effective writing. 
The evidentiary base of the resulting writing, therefore, is 
not often robust. 

Institutional Placement Tests 
In contrast, most of the institution-level placement tests 
reviewed for this study include one or more reading 
passages and call on students to marshal evidence from 
the passages to argue a position or compare and contrast 
differing positions. The student is directed to refer to 



the passage in his or her response and often is asked to 
explain the author’s point of view. The reading passages 
range from 65 to 800 words and are similar in com- 
plexity to the Level 5 and 6 reading passages discussed 
previously. By providing evidence to adjudicate, these 
writing tests push students to read, synthesize, analyze 
and respond to material that may be unfamiliar to them 
— very similar to the challenges they will face in their 
college courses. 

Do the scoring rubrics and anchor papers 
examined reflect the college-ready writing 
expectations of institutions of higher 
education? 

One way that writing assessments communicate the level 
of writing required is through accompanying scoring 
rubrics and anchor papers. Scoring rubrics provide a 
framework for considering the effectiveness of a student’s 
composition by providing a range of levels — usually 
six — from lowest to highest and describing the traits of 
writing at each of those levels. Anchor papers are actual 
samples of student writing at each level. They are used 
to show the level of writing required to reach each 
score point. 

Admissions and Placement Tests 
The scoring rubrics and sample papers for most of the 
admissions and placement tests examined in this study 
emphasize characteristics of strong writing, such as the 
ability to take a position and use evidence to present a 
point. They also tend to focus on proper use of grammar 
and syntax and the ability to use language effectively and 
fluently to express ideas. They underscore the importance 
of analytic skills and the writer’s ability to reason. 

For example, the ACT rubric describes an essay with the 
highest score of six as one that “addresses complexity 
by examining different perspectives on the issue, or by 
evaluating the implications and/or complications of the 
issue, or by fully responding to counterarguments to the 
writer’s position.” Similarly, the SAT rubric describes 
a top essay as one that “effectively and insightfully 
develops a point of view on the issue and demonstrates 
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TYPES OF WRITING AND APPROACHES ON DIRECT WRITING TESTS 





Test/Institution 


Requires Students 
To Take a Position 


Provides Students 
with Supporting 
Evidence To Use 


National 

Admissions 

Tests 


ACT 


SAT 


National 

Placement 

Tests 


ACCUPLACER WritePlacer 


COMPASS e-Write 


State or 

Systemwide 

Tests 


California 


■ 


■ 


Texas 


Institution- 
Level Tests 


College of New Jersey 


■ 


■ 


Louisiana State University 




■ 




Purdue University-North Central 




Temple University 


■ 


■ 




University of Maryland-Baltimore County 




University of Massachusetts-Amherst 


■ 


■ 




Youngstown State University 


■ 


■ 



outstanding critical thinking, using clearly appropriate 
examples, reasons and other evidence to support its 
position.” Slight variations on this are included in all of 
the scoring guides examined. 

Overall, these rubrics and sample papers reflect rela- 
tively sophisticated writing in which students effectively 
present a position and supply evidence. These are skills 
that will serve them well in college and beyond. In many 
cases, the scoring rubrics and sample papers are avail- 
able publicly via the Web, providing an opportunity for 
schools and teachers to help students better understand 
the expectations. 



Indirect Writing Assessments 

Direct writing tests are the best way to assess students’ 
overall writing abilities, while indirect writing tests are 
better for assessing isolated skills. Nonetheless, although 
indirect writing tests may be insufficient for assessing 
students’ ability to produce written text, they can be 
effective in assessing specific elements of the writing 
process, such as the fundamentals of editing and revision. 
Grammar, usage, sentence structure and punctuation 
can be assessed indirectly, as can more sophisticated 
aspects of revision such as organization, strategy and 
word choice. A good indirect test of college-ready writing 
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ought to measure at high levels the full range of writing 
mechanics and structure valued by college faculty. 

For this study, Achieve analyzed five indirect writing 
tests: the ACT and SAT admissions tests, a sampling 
of items from the computer-adaptive ACCUPLACER 
Sentence Skills and COMPASS Writing Skills place- 
ment tests, and the Texas Higher Education Assess- 
ment. Achieve ’s experts coded each item on each test 
on the basis of content and cognitive demand. The tests 
incorporate more than 600 multiple-choice items that 
are included either as part of a general writing test or as a 
complement to the reading items in the admissions and 
placement tests. 

What skills are measured by indirect writ- 
ing tests, and what do they reveal about a 
student's ability to write? 

To be successful writers, students will need to use sophis- 
ticated vocabulary; appropriate and varied sentence struc- 
ture; and coherent, developed paragraphs. Additionally, 
college professors cite correct grammar, usage, punctua- 
tion, capitalization and spelling as critically important. 
Although these abilities are best measured through actual 
writing, indirect writing tests do measure these valuable 
skills in isolation and can serve as approximations of a 
student’s writing ability. 

All of the questions on the indirect writing tests exam- 
ined in this study were identified as focusing on either 
editing or revision. Editing questions address issues of 
sentence structure, grammar and usage, while revi- 
sion questions focus on rhetorical skills such as writing 
strategy, organization and style. All of these skills are 
important for successful preparation for college writ- 
ing. However, questions that focus on word or sentence 
correctness can assess only specific applications of rules 
while revision items depend more on an understanding 
of the whole text. Therefore, generally speaking, revision 
questions challenge students’ logic and reasoning in a 
way that demands the critical thinking skills valued by 
college faculty. 



TESTS ANALYZED IN INDIRECT WRITING 



Test Category 


Indirect Writing 


National 


■ 


ACT Assessment — English 


Admissions 




(two forms) 


Tests 


■ 


SAT Reasoning Test — 
Writing (two forms) 


National 


■ 


ACCUPLACER — Sentence 


Placement 




Skills 


Tests 


■ 


COMPASS — Writing Skills 


State or 


■ 


Texas Higher Education 


Systemwide 

Tests 




Assessment 



Admissions Tests 

Each of the admissions tests of indirect writing targets a 
range of writing skills, from editing for punctuation and 
grammar errors to concerns with organization and use 
of details. However, considerable differences in emphasis 
exist. The SAT predominantly targets editing skills — 85 
percent — while the ACT addresses editing and revision 
almost evenly. In large part, this difference in emphasis 
is the result of their different formats. The ACT test uses 
full passages, which enables questions about revisions 
to tone, style and organization. The SAT, on the other 
hand, uses sentences more often than full passages, 
which invites questions about editing only at the 
sentence level. 

Placement Tests 

Interestingly, these priorities are reversed in the 
two companies’ placement tests: ACT’s COMPASS 
focuses slightly less than three-fourths of its items on 
editing, while nearly two-thirds of College Board’s 
ACCUPLACER measures revision skills. 

How cognitively challenging are the 
indirect writing test questions? 

To analyze these tests, Achieve ’s experts coded the ques- 
tions along a continuum of cognitive demand similar to 
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that used for reading. Items that require 
only knowledge of a rule, such as choos- 
ing the correct form of a verb or correct- 
ing a sentence fragment, were coded as 
literal recall. If a test taker had to apply 
some understanding of the meaning of 
the sentence or text to correctly answer 
a question — or consider the impact of 
a particular device on the meaning of 
the sentence or passage — the item was 
coded at low inference or high infer- 
ence. For example, items that required 
choosing a word appropriate for the 
tone of a text were coded as low infer- 
ence; items requiring the determination 
of whether or not to include or delete 
information from a text, based on an 
understanding of how that decision 
would affect meaning, were coded as 
high inference. Analysis was reserved 
for situations in which the test taker had 
to analyze a passage to determine the 
correct answer. 

Admissions and Placement Tests 
The vast majority of editing items 
analyzed from the admissions and 
placement tests rely only on recognizing 
and applying a memorized rule about 
usage or mechanics; therefore, they 
were coded at the lowest level of cogni- 
tive demand — literal recall. Collec- 
tively, there appears to be an emphasis 
on lower-level cognitive skills, lacking 
the ability to reveal a fuller sense of 
students’ abilities. 



Skills Measured on Indirect Writing Tests 
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Cognitive Demand of Indirect Writing Tests 
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Overall, how rigorous are the 
indirect writing tests? 

To gauge the overall challenge of the 
admissions and placement indirect 
writing tests, Achieve used a scale that 
ranged from low challenge (Level 1) 
to high challenge (Level 6). This scale 
was based on the ACT assessment 
frameworks for tests administered to 
students in grades 8-9, 10 or 11-12. 
The level of an item takes into account 
both what the item is measuring (for 
example, punctuation or organization) 
and the difficulty of that skill based on 
empirical evidence from thousands of 
test takers (for example, from using a 
comma in a series to using a semico- 
lon to indicate a relationship between 
closely related independent clauses). 

On this scale, a test for students in 
grades 11 and 12 theoretically should 
include a heavy component of Level 5 
and 6 questions and a smaller compo- 
nent of questions at Levels 1 and 2, 
which tend to be middle school level. 



Level of Challenge of Indirect Writing Tests 




ACT SAT ACCUPLACER COMPASS 



Admissions 



Placement 



I I Level 1 □ Level 3 □ Level 5 
El Level 2 LJ Level 4 ■ Level 6 



Note: Scale ranges from Level 1 (low challenge) to Level 6 (high challenge). 
Totals may not equal 100 percent due to rounding. 



sentence-combining techniques to improve flow. 
Students frequently will use these skills in their college 
writing assignments. 



Admissions Tests 

Overall, the ACT and SAT are relatively rigorous: 59 
percent and 71 percent, respectively, of the questions on 
the ACT and SAT tap skills at Levels 3-6. The ACT 
is particularly challenging with nearly 20 percent of 
its items at Levels 5 and 6. Questions at the upper end 
of the scale might require the student to use logic in 
thinking through the process of reorganizing a para- 
graph to accomplish a specific purpose or to use 



Placement Tests 

The vast majority of the questions on the placement 
tests fall at the lower levels of the scale (Levels 1 and 2). 
A typical Level 2 question asks students to choose the 
correct possessive pronoun (“its” versus “your”), use the 
appropriate spelling of “its/it ’s” or check for subject-verb 
agreement. These are skills that ideally students should 
master well before high school. 
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Findings: Mathematics 



SUMMARY 

To be successful in college and on the job, high school graduates need to be able to use 
mathematics to solve challenging problems. Achieve's ADP research with college professors 
from diverse institutions found that incoming freshmen need to have mastered the content 
taught in a four-year course sequence that includes courses such as Algebra I, Algebra II and 
Geometry, as well as significant data analysis and statistics. 

Achieve's analysis of admissions and placement tests found that: 

■ The admissions and placement tests put their heaviest emphasis on algebra — content 
that is important to colleges and high-skilled workplaces. However, the algebra content 
assessed tends to favor prealgebra and basic algebra over the advanced algebra concepts 
and skills essential for college readiness and placement into College Algebra. 

■ Placement tests in particular are narrow and do not reflect the full range of content 

— such as data analysis, statistics and geometric reasoning — college students need in a 
wide variety of courses. Admissions tests are more balanced across topics. 

■ Too few questions on admissions and placement tests tap higher-level cognitive skills 
critical to success in college. 



In contrast to the unique role of Comp 101 as the 
nearly universal first credit-bearing course in English, 
most colleges offer students several options for their 
first credit-bearing course in mathematics, the most 
common of which is College Algebra (see discussion on 
page 8). Other courses include Elementary Statistics, 
Mathematics for Liberal Arts, 13 Mathematics for 
Elementary Teachers and Calculus. 

With the exception of Calculus, these courses build on 
the knowledge and skills normally acquired in three 
years of high school mathematics. College Algebra and 
Elementary Statistics make extensive use of techniques 
taught in Algebra II, while other introductory credit- 
bearing mathematics courses depend less on mastery of 
techniques than on mathematical habits of mind. With 
few exceptions, college courses that cover topics normally 
taught in high school Algebra II or earlier, such as Inter- 
mediate Algebra, do not carry credits that count toward 
an associate or a bachelor’s degree. 



For this study, Achieve analyzed a variety of mathematics 
tests that are used for placement purposes by colleges and 
universities. These tests include examples of four differ- 
ent types: national admissions tests, national placement 
tests, state or systemwide tests and institution-level tests 
(see table on the next page). In addition, included in the 
analysis was a small pool of science items identified by 
ACT analysts as assessing mathematics content as it 
arises in natural applications. Although mathematics- 
intensive items on the ACT science test are incorporated 
into a composite score useful in making college admis- 
sion decisions, such items are not reported as part of the 
ACT mathematics score and, therefore, are not incor- 
porated into decisionmaking about course placement in 
mathematics. 

As shown on page 27, Achieve ’s analysis of placement 
tests included those used for determining placement into 
College Algebra and Calculus. Because College Algebra 
is the most common introductory college mathematics 
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TESTS ANALYZED IN MATHEMATICS 



Test Category 


Mathematics: College Algebra 


Mathematics: Calculus 


National Admissions 
Tests 


ACT Assessment — 
Math (two forms) 








SAT Reasoning Test — 
Math (two forms) 






National Placement 
Tests 


ACCUPLACER — Elementary 
Algebra 


■ 


ACCUPLACER — College Level 
Mathematics 




COMPASS — Algebra 


■ 


COMPASS — College Algebra 


State or Systemwide 
Tests 


Texas Higher Education Assessment 

Washington Intermediate 
Mathematics Placement Test 


■ 


Washington Advanced 
Mathematics Placement Test 


Institution-Level Tests 


Mathematical Association of 
America Algebra Placement Test 


■ 


Louisiana State University College 
Algebra Placement/Credit Exam 




Purdue University-North 
Central Student Assessment and 
Measurement Placement Test 


■ 


Mathematical Association of 
America Calculus Readiness 
Placement Test 




Temple University Mathematics 
Placement Test 


■ 


University of Maryland-College 
Park Math Placement Test 




University of Minnesota College 
Math Readiness Test 


■ 


University of Minnesota Calculus 
Readiness Test 


Achieve's analysis focuses on the tests used for placement into College Algebra; for an analysis of the tests used for 
placement into Calculus, see sidebar on page 33. 



course taken at two- and four-year colleges, the focus of 
this report is on the first-tier tests. Those are the tests 
that differentiate students who are ready for a credit- 
bearing College Algebra course from students in need of 
remediation. 

Achieves experts analyzed more than 1,200 items 
(including 22 ACT science items), coding them for 
their mathematical content and cognitive demand (e.g., 
whether students are called upon to recall knowledge, 
reason or generalize). In addition, using a taxonomy 
developed from the Third International Mathematics 
and Science Study (TIMSS), Achieve was able to deter- 
mine a typical grade level for each item. This level, called 



an International Grade Placement (IGP) index, can be 
thought of as a kind of composite among the 40 TIMSS 
countries (other than the United States) to show when 
the curriculum focuses on different mathematics content 
— at which point the highest concentration of instruc- 
tion on a topic occurs. The average international grade 
level for each test then is calculated by averaging the IGP 
indices of each item. 

Achieve’s investigation revealed both predictable and 
surprising features of these tests. Because national 
admissions tests tend to sample all mathematics taught 
in the first three years of high school, they typically are 
broader with respect to content coverage and sometimes 
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are more cognitively challenging 
than tests designed specifically for 
placement purposes. In contrast, 
placement tests generally focus on 
a narrower range of knowledge 
and often employ less cognitively 
demanding skills to determine 
which students are likely to suc- 
ceed in particular courses. 

Because most colleges and uni- 
versities offer entering students 
four or five different mathematics 
courses requiring different levels 
of mathematical preparation — 

College Algebra being the most 
common — some placement tests 
are arranged in multiple tiers, 
with topics ranging from arith- 
metic to precalculus. Even though 
this study focuses primarily on 
placement tests that are used to 
place students into College Alge- 
bra, Achieve also examined tests 
designed to indicate readiness for 
Calculus. 

Do college admissions and placement tests 
measure the full range of knowledge and 
skills described in the ADP benchmarks? 

ADP found that, to meet the demands of the many 
different university and college programs, high school 
graduates need to master a broad range of mathematical 
skills that are particularly important to professors from a 
variety of disciplines. In addition to algebraic concepts, 
professors identify geometry, measurement, data analysis 
and statistics as vital for success in credit-bearing courses. 
These topics are included in the ADP benchmarks and 
in most state standards. Achieve analyzed the admissions 
and placement tests to determine the distribution of 
topics, including algebra, geometry/measurement, data 
analysis/statistics and number concepts. 



Admissions Tests 

Both the ACT and the SAT mathematics tests place the 
greatest emphasis on algebra, with nearly half of their 
collective items assessing algebraic content. Secondary 
emphasis is placed on geometry, with more than 30 per- 
cent of the questions across the ACT and SAT addressing 
geometry and measurement. Substantially less emphasis 
is given to number concepts (14 percent) and to data 
analysis and statistics (7 percent). 14 This distribution of 
math content is more balanced than that of placement 
tests. 

Placement Tests 

Achieve’s analysis shows that three-quarters of the 
items on the placement tests measure algebra. 15 Tests 
designed for placement into College Algebra courses 
collectively attribute only 13 percent of their items to 
geometry and measurement, with institution-level tests 
providing somewhat greater emphasis than national 
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tests. Treatment of data and statistics 
is particularly lacking on placement 
tests, with only 1 percent of items 
assessing these concepts. Assessment 
of number sense and numerical 
operations also is minimal on 
placement tests, with 1 1 percent of 
items collectively addressing such 
topics. Emphasis on number concepts 
is slightly more prevalent on state and 
institution-level placement tests than 
on national placement tests. 

How challenging is the 
algebra measured by the 
admissions and placement 
tests? 

As previously indicated, what is 
considered a credit-bearing course 
in mathematics varies by major and 
across institutions. College Algebra 
— which also is called Precalculus 
because it bridges Algebra II and 
Calculus — is generally accepted as 
a transfer-worthy, credit-bearing col- 
lege course; it is at this point where 
course placement becomes critical for 
students entering college. In theory, 
high school students should be able 
to demonstrate mastery of Algebra 
II-level content to be placed into a 
College Algebra course. Achieve ’s 
detailed coding of the admissions and 
placement test items made it possible 
to group the many algebra items into 
levels that approximately correspond 
to typical courses: “prealgebra” 
(middle school), “basic algebra” 
(Algebra I), and “advanced algebra” 
(Algebra II). See the table to the right 
for more details. 



ACHIEVE ALGEBRA LEVELS 



Algebra Level 


Course Equivalent 


Examples of Topics 
Covered 


Prealgebra 


Middle School 


Working with 




Math/Prealgebra 


integers, rational 
numbers, patterns, 
representation, 
substitution, basic 
manipulation and 
simplification 


Basic Algebra 


Algebra 1 


Linear equations and 
basic functions and 
relations 


Advanced 


Algebra II 


Non-linear equations 


Algebra 




or inequalities and an 
understanding of real 
and complex numbers 
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Admissions Tests 

Nearly one-third of the ACT and SAT collectively assess 
more rigorous advanced algebra concepts and skills typi- 
cally encountered in a course such as Algebra II. Most 
of the advanced algebra items found on the admissions 
tests address solving quadratic equations and systems of 
equations. Of the remaining items, slightly more than 25 
percent of the ACT and SAT assesses basic algebra topics 
typically encountered in courses such as Algebra I, and 
the remaining 40 percent covers prealgebra concepts. 
Although the ACT and SAT are more rigorous than most 
of the placement tests with respect to algebra, many 
important higher-level algebra topics included in the 
ADP benchmarks — such as advanced functions and 
modeling — are barely addressed on either of these tests. 

Placement Tests 

On average, the items Achieve 
analyzed from the placement tests 
emphasize prealgebra knowledge 
and skills. Fewer than one in 
four items assess knowledge and 
skills typically encountered in 
courses like Algebra I. Less than 
30 percent of the algebra items 
across all placement tests tap 
advanced algebra knowledge and 
skills typically encountered in a 
course such as Algebra II. There 
are a few notable exceptions: The 
Minnesota and Mathematical 
Association of America (M AA) 

Algebra tests actually place their 
greatest emphasis (50 percent of 
each) on advanced algebra knowl- 
edge and skills. More so than on 
the admissions tests, nearly all 
of the advanced algebra items on 
the placement tests — including 
the Minnesota and MAA Algebra 
tests — ask students to solve 
quadratic equations and systems 
of equations. 



How cognitively challenging are the 
mathematics test questions? 

The content measured by an item reveals only one aspect 
of a test’s character. A more complete understanding also 
requires consideration of the cognitive demand of the 
questions. For example, are students asked to develop a 
mathematical model to solve a complex, multistep prob- 
lem? Or is the question framed in such a way that students 
need apply only a basic procedure, such as solving a simple 
linear equation? To analyze cognitive demand, Achieve 
used a five-point scale with Level 1 (recalling) being the 
lowest and Level 5 (advanced reasoning) the highest. 

Admissions Tests 

The ACT and SAT are the most cognitively demanding 
of all the tests examined in this study, with nearly 40 per- 
cent of their items categorized as Level 3 or above. Level 
3 items require students to use non-routine procedures, 
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Level 4 items ask students to formulate problems and 
strategize solutions, and Level 5 items call on students 
to use advanced reasoning. Although mathematics- 
intensive items on the ACT science test are not counted 
for mathematics placement purposes, they are particu- 
larly noteworthy because Achieve ’s analysis identified all 
of these items as Level 3 or 4, given the cognitive com- 
plexity of reading, analyzing and interpreting scientific 
data. In addition, it is worth mentioning that nearly 20 
percent of items on the SAT are gridded-response items 
in which no answer choices are given. Students must 
produce their own answers to these questions and bubble 
in their answers on a special answer grid. The SAT is the 
only mathematics test in this study that includes such 
items; the remaining tests rely exclusively on multiple- 
choice questions. 

Placement Tests 

Achieve ’s analysis showed that place- 
ment tests do not tap the higher 
levels of cognitive demand. In fact, 
collectively across all placement tests 
examined, more than three-quarters 
of the questions are categorized as 
Level 2 — calling for students to apply 
routine procedures to solve problems. 

Seventeen percent of the questions fall 
at Levels 3 and 4. Only two questions 
(on the Texas assessment) across all of 
the placement tests call for students to 
use advanced reasoning (Level 5), even 
though research indicates it is neces- 
sary for success in college. Interestingly, 
state and institution-level placement 
tests on average tend to be more cog- 
nitively demanding than the national 
placement tests, placing greater empha- 
sis on items rated as Level 3 or above. 

The Washington Intermediate Math- 
ematics Placement Test and the Texas 
assessment are particularly notable for 
their considerable emphasis on assess- 
ing higher levels of cognitive demand. 



How rigorous is the content on the col- 
lege admissions and placement tests on an 
international scale? 

As part of its earlier study of high school exit tests. 
Achieve examined the International Grade Placement 
(IGP) of each item on each test. The IGP, an index devel- 
oped by Michigan State University, represents the average 
grade in which a mathematics topic typically appears in 
the curriculum of the 40 nations (other than the United 
States) that took part in the Third International Math- 
ematics and Science Study (TIMSS). The IGP rating of a 
test is calculated by averaging the IGP levels of the items 
on the test. TIMSS data show that students in other 
countries typically study mathematical topics a year ear- 
lier than in most U.S. school districts; therefore, the IGP 
index is approximately one year lower than correspond- 
ing U.S. averages. Ideally, admissions and placement tests 
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would place greater emphasis on content taught later in 
high school and less emphasis on prealgebra. 

Admissions and Placement Tests 
Achieve ’s analysis indicates that although some questions 
on admissions and placement tests do measure advanced 
content, overall they place too much emphasis on content 
that is taught earlier in a student’s high school career. 

The average IGP for the set of items from all of the 
institution-level placement tests is 8.4, indicating that — 
on average — the tests focus on content that is taught in 
the 8th grade in other countries but in early high school 



in the United States. The average IGP for the national 
admissions and placement tests are comparable to the 
institution-level tests. This is largely because these tests 
include so many items that measure prealgebra and basic 
algebra content. As the box-and- whisker plot on page 31 
indicates, the tests do measure some higher-level knowl- 
edge and skills that are prerequisites for success in college 
(as evidenced by the “whisker” extending upward toward 
the later high school years), but they tend to be weighted 
with large proportions of items categorized as assessing 
content at the 9th grade level or below internationally. 
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Advanced-Level Placement Tests 



Overall, Achieve ’s analysis found that most placement 
tests for College Algebra measure relatively low-level 
content. Achieve also examined several advanced-level 
placement tests designed to place students beyond a 
first-year, credit-bearing course such as College Algebra 
— sometimes into Calculus. These advanced tests are 
similar to the basic-level tests in that they all emphasize 
algebra. They are different, however, in that they assess 
more advanced levels of algebra, including concepts such 
as non-linear functions and equations — the kind of con- 
tent Achieve expected to see more of in the placement tests 
intended to determine readiness for the first credit-bearing 
college mathematics course. 

Collectively, the algebra items from the advanced-level 
tests examined by Achieve emphasize advanced algebra 
knowledge and skills, with 57 percent of the items 
assessing algebra at this level. Of the institution-level 
tests, the Louisiana State University College Algebra 
Placement/ Credit Exam is particularly noteworthy for the 
large proportion of its algebra items (two-thirds) devoted 
to advanced algebra — probably because this test is used 
not only to place students but also to award credit 
for and placement out of College Algebra. 



ADVANCED-LEVEL TESTS ANALYZED IN MATHEMATICS 



Test Category 


Mathematics: Calculus 
Placement 


National 


■ 


ACCUPLACER — College Level 


Placement 




Mathematics 


Tests 




■ 


COMPASS — College Algebra 


State or 


■ 


Washington Advanced Mathematics 


Systemwide 




Placement Test 


Tests 


Institution- 


■ 


Louisiana State University College 


Level Tests 




Algebra Placement/Credit Exam 




■ 


Mathematical Association of America 
Calculus Readiness Placement Test 




■ 


University of Maryland-College Park 
Mathematics Placement Test 




■ 


University of Minnesota 
Calculus Readiness Test 



Algebra on Calculus Placement Tests 



Overall, these higher-level tests are a better reflec- 
tion of the mathematics that professors say are 
important for success in postsecondary endeavors. 
Nevertheless, the focus of the advanced tests is 
narrow, with a majority of the items covering alge- 
bra and some trigonometry. And like the regular 
placement tests, these advanced tests continue to 
call for a low level of cognitive demand, with little 
beyond the routine application of procedures. 
Thus, despite their coverage of more advanced 
content, these tests are unable to fully capture the 
breadth of content and skills needed by students 
if they are to be successful in the range of courses 
they will likely take in college. 




f 3 I 

National Placement into College Calculus 



1 I Prealgebra H Basic Algebra H Advanced Algebra 



Totals may not equal 100 percent due to rounding. 
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Recommendations for Policymakers 

High school students across the country take a lot of tests — particularly students planning 
to go to college. But as this study of college admissions and placement tests and Achieve s 
earlier study of high school graduation exams reveal, high school and college exams send 
mixed signals about what students need to learn to be successful. If states are going to 
improve the preparation of high school students so that they graduate prepared for college, 
states will need a more rigorous and coherent system of assessments — one that is capable of 
measuring the standards students are expected to meet in high school and signaling readiness 
for postsecondary success. 



A number of strategies are emerging as states work to 
build admissions and placement tests into their high 
school assessment and accountability systems. Some 
states are incorporating the ACT or SAT into their 
existing systems and requiring all students to take those 
exams. Others are considering using college placement 
tests as early indicators to signal to students whether 
they are ready for credit-bearing college courses. A 
growing number of states are developing end-of-course 
tests that align with their high school curriculum. Some 
are modifying their high school tests to make them 
better measures of college readiness and some are setting 
“college-ready” cut scores on existing tests. 

Although it is too early to see which strategies hold 
the most promise for improving student preparation, 
whichever path states pursue, the assessments must meet 
two important goals: They must adequately measure 
college readiness, and they must be well aligned with the 
state high school standards. If either goal is sacrificed, 
the validity of the assessments will be compromised, and 
their impact on student preparation will be limited. With 
a growing number of states working hard to align their 
high school graduation standards with expectations of 
college faculty and employers, it is important for states 
to re-examine their high school and college assessments 
to ensure that the assessments clearly and consistently 
measure students’ college readiness. 



Recommendations for 
K-12 Policymakers 

Augment admissions tests when incorporating them 
into statewide testing systems. A number of states are 
incorporating national college admissions tests — the 
ACT and SAT — into their high school assessment and 
accountability systems. They are requiring all students, 
not just the college bound, to take these tests. The argu- 
ment is that the tests are widely used by colleges and 
universities for admissions and placement and therefore 
have credibility with students, parents and the general 
public. Giving all students the opportunity to take the 
tests will give them a leg up on the college preparation 
process and may encourage students who did not view 
themselves as college material to pursue postsecondary 
education. Because most college-bound students already 
take these tests in high school, incorporating them into 
a state’s assessment system will not significantly increase 
the test burden. 

However, states taking this path should proceed with 
caution. Achieves analysis reveals that although 
admissions tests do some things very well, there are gaps 
in what they measure. Neither the ACT nor the SAT 
includes the full range of advanced concepts and skills 
reflected in the ADP benchmarks and, increasingly, in 
state high school standards. Therefore, simply incorpo- 
rating these tests as they are into state high school testing 



Achieve. Inc. 




systems is an insufficient strategy. To be effective, states 
need to augment the ACT and SAT with additional test 
questions or with additional performance measures to 
ensure stronger alignment with state standards and to 
assess the more advanced concepts and skills that college 
faculty say are important. 

States are approaching augmentation in different 
ways. Maine, for example, is administering a slightly 
augmented SAT to all 11th grade students. The state 
worked with the College Board to develop supplemental 
items in data and statistics — an area of mathematics 
that is covered in the state standards but not extensively 
assessed on the SAT. Similarly, Michigan administers 
the ACT and WorkKeys to all 11th graders, along with 
15 additional mathematics items that measure content 
not covered on either test. Kentucky plans to administer 
both the ACT and the state assessment to all students. To 
reduce testing time, the state has analyzed both tests and 
will eliminate items from the state test that cover content 
measured by the ACT. 

ACT and the College Board have shown great leadership 
in establishing standards for the transition from high 
school to college, and both organizations are actively 
involved in increasing not only the college-going rate, but 
also the level of student preparation for college success. 
Achieve encourages states that are considering incorpo- 
rating the ACT or SAT into their state assessment and 
accountability systems to take stock of what these tests 
measure and how well aligned they are with state stan- 
dards. Then, states can work with ACT and the College 
Board to augment the assessments as needed to ensure 
greater coherence and alignment. 

Consider using end-of-course tests to tap higher-level 
content and skills and to place students into college 
courses. A growing number of states are pursuing end- 
of-course tests in high school as a strategy for measuring 
college readiness while also better aligning tests with 
the high school curriculum. There are multiple benefits 
to this approach. First, the tests can be tied closely to 
the curriculum and to the courses that states require 



students to take to graduate. Second, end-of-course 
tests are more sensitive to instruction because they are 
taken right after a student completes a course. For these 
reasons, end-of-course tests can play the unique dual role 
of indicating student readiness and providing teachers 
with information to help them improve the curriculum 
and instruction. End-of-course tests also allow states to 
monitor performance and ensure consistency of rigor in 
courses taught across the state. 

For end-of-course tests to serve as an indicator of college 
readiness, states will need to administer tests in higher- 
level courses, such as Algebra II and 11th or 12th grade 
reading and writing. Most states, including those with 
end-of-course assessments, do not currently have tests 
at this level. It also is essential for higher education to 
play a role in developing and/or reviewing these exams 
to ensure they reflect the skills needed for college suc- 
cess. If higher education participates in developing these 
exams, they can more readily use the tests for placement 
into entry-level college courses. This will send a power- 
ful signal to students and their parents and teachers that 
performance in high school pays off in college. 

There are challenges associated with end-of-course test- 
ing. First, this approach could increase the amount of 
testing required by a state, depending on the number of 
courses for which the state develops an assessment. Sec- 
ond, colleges may be willing to use end-of-course tests for 
placement only if they are administered in 11th or 12th 
grade. In the case of students taking Algebra II in the 
10th grade — two years before entering college — higher 
education institutions would need to determine whether 
results could be used for placement or whether additional 
information (e.g., evidence of continued course-taking 
in advanced mathematics or additional tests) would be 
needed. 

Modify existing high school tests to measure college 
readiness. Some states are considering adding ques- 
tions to existing high school assessments to round out 
those tests and make them adequate measures of college 
readiness. California’s education department took this 
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approach, working with the California State University 
(CSU) system and adding a short set of questions to the 
11th grade standards-based tests in English and math- 
ematics to measure the additional skills CSU faculty say 
incoming freshmen need to be successful in entry-level 
courses. Students who choose to take the CSU portion 
of the 11th grade test and score well on it are exempt 
from the CSU-required mathematics and English place- 
ment tests, as long as they continue to take challenging 
courses in their senior year of high school. The modified 
11th grade exams also serve to alert students if they need 
additional preparation for college in time to adjust their 
senior year coursework. 

Texas is another example. Since fall 2003, the Texas 
Success Initiative — created by the state Legislature and 
supported by the Texas Higher Education Coordinat- 
ing Board — has allowed scores on the state 11th grade 
test (the TAKS) to be used for placement into freshman 
composition and mathematics courses. This is the same 
11th grade exam that is used for graduation from high 
school. Students need to earn a certain score to gradu- 
ate and a higher score to enter credit-bearing courses in 
Texas postsecondary institutions. 

Of course, this strategy would be very difficult to pursue 
in states with 10th grade tests or tests that measure 
minimum, rather than college readiness, standards. It 
is more appropriate for states that give 1 1th grade tests, 
such as California and Texas, so they can measure more 
advanced content. 

Use existing college placement tests for diagnostic 
purposes only. Some states, such as Ohio, are making 
their college placement tests available for students to 
take voluntarily in high school. These exams provide 
information to high school students about their readiness 
for credit-bearing, first-year college courses and allow 
teachers to work with students to address learning gaps 
in their senior year so that students are better prepared 
when they get to college. 

This use of placement tests as an early signal of readiness 
makes sense, particularly if it can be used to engage 



students who might not otherwise be considering college 
and to shore up students’ skills. It is not, however, a 
substitute for building more comprehensive measures 
of college and work readiness into the state high 
school accountability system. The math placement 
tests reviewed by Achieve are more narrowly focused 
on algebra than the admissions tests, which measure 
a broader balance of content. They also tend to be 
less rigorous than admissions tests, as they fail to tap 
a number of the higher-level skills found in the ADP 
benchmarks. As a result, if these tests are used for school 
accountability purposes, they could narrow and water 
down the curriculum. Therefore, Achieve does not 
recommend that states consider incorporating these 
assessments into their high school accountability systems 
without supplementation or adaptation. 

The Role of Higher Education 
Policymakers 

Postsecondary leaders have a critical role to play in 
developing a coherent system of high school assessments 
capable of measuring college readiness. In addition, 
although colleges and universities will continue to 
use admissions and placement tests for the important 
purposes they serve, they should not allow these tests to 
serve as a de facto standard of college readiness. Rather, 
they should clearly communicate their expectations and 
work with the K-12 leaders and the testing community 
to help ensure that college admissions, college placement 
and high school tests are more aligned with these expec- 
tations. Over time, as high school tests become better 
indicators of readiness for college, institutions should 
consider using them as a factor in placement decisions, 
streamlining the transition to college and reducing the 
number of tests students take overall. 

Clearly define expectations for incoming students. 

Through ADP, Achieve has discovered that postsecond- 
ary faculty have a fairly consistent view of the mathemat- 
ics and communications skills that incoming freshmen 
need to be successful in first-year, credit-bearing college 
courses. In mathematics, faculty reported that students 




need knowledge and skills typically learned in a four- 
year mathematics sequence including Algebra I and II, 
Geometry, data analysis, and statistics. They also need 
sophisticated mathematical reasoning and problem- 
solving skills. In English, faculty stated that students 
need to be able to write and communicate effectively to 
different audiences, they need to be able to understand 
and analyze various types of complex informational 
texts, and they need to apply sophisticated analytic and 
reasoning skills. However, these expectations are rarely 
articulated publicly, nor are they reflected in the place- 
ment tests analyzed by Achieve; they reside in profes- 
sors’ minds and are reflected in their syllabi and course 
assignments. These expectations need to become more 
widely known by and communicated to K-12 educators, 
students and their parents. 

Postsecondary systems and institutions must be clearer 
and more transparent about what it means to be college 
ready so that the K-12 system has something tangible 
to aim for. In each state, colleges and universities — in 
partnership with the K-12 system — should collaborate 
to define and publicize statewide standards for transition 
from high school to college. These standards should be 
pegged to the knowledge and skills high school graduates 
need to succeed in credit-bearing, non-remedial courses. 
They must be concretely articulated, much like K-12 
academic content standards, so that they do not simply 
become represented by a score on an admissions or a 
placement test. And, as stated earlier, higher education 
should align admissions and placement tests to these 
articulated readiness standards. 

Work to align expectations and define new standards for 
transition to college is under way in a growing number of 
states. These efforts hold great promise and should lead 
to more coherent and aligned assessments. 

Scrutinize placement tests given to incoming students 
to determine eligibility for entry into credit-bearing 
courses. Once postsecondary institutions have clearly 
defined their expectations, it is important that they 
examine their existing placement tests (and admissions 



tests, if they are used to make placement decisions) to 
see whether they measure the full range of content and 
skills important for success in college. Often, placement 
tests have been in place for a long time, either selected or 
created by previous faculty and rarely re-examined. In 
some cases, the tests may be pegged to the existing level 
of preparation of many students entering open admis- 
sion or broad access institutions, rather than to the level 
they will need to be fully prepared for success once they 
arrive. 

To ensure that placement tests are relevant and aligned 
with today’s expectations, institutions should convene 
faculty from across the arts and sciences to analyze the 
placement tests to see whether they measure the con- 
tent and skills necessary to enter and succeed in college 
— not just to avoid remediation. The faculty also should 
look at the cut scores on the tests to see whether they set 
meaningful targets for students. If, as Achieve found, 
the tests do not fully measure the rigor and breadth of 
college readiness expectations, college faculty should 
consider commissioning or developing placement tests 
that do. Although it is likely that there will always be a 
need for tests that determine whether remediation is nec- 
essary, colleges should work to ensure that the tests used 
to indicate readiness for credit-bearing courses actually 
reflect the real demands of what it takes to succeed in 
those courses. By setting a clear and accurate target, such 
tests can help improve student preparation in a way that 
less demanding tests cannot. 

Collaborate with K-12 on the development of high 
school tests that fully reflect the breadth and rigor of 
the content needed for success in postsecondary edu- 
cation. In addition to incorporating more challenging 
placement tests, colleges also should work with K-12 to 
develop stronger high school tests. If states are building 
systems of end-of-course tests, postsecondary institu- 
tions should collaborate with them on the full range of 
tests — not just those at the end of high school, such as 
Algebra II or 1 1th grade English — to ensure that they 
cumulatively measure what it takes to succeed in college. 
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Colleges should use the results on a subset of the tests to 
place students in courses appropriate to their knowledge 
and skills. In addition, colleges should align their cut 
scores and establish a single standard that applies no mat- 
ter where a student decides to go to college. Coordina- 
tion of this type between the higher education and K-12 
systems would have two distinct benefits: It would reduce 
confusion about what is required for college -level work, 
and it would do so in a way that reduces the number of 
tests a student takes. 

Support the development and use of K-16 longitu- 
dinal data systems. Institutions of higher education 
need to work with K-12 systems to develop effective data 
systems that follow students from high school to college. 



Without data on how students perform once they arrive 
in college — for example, their need for remediation and 
their success in first-year courses — no one can be sure 
which high school programs are effective and which need 
improvement. Because the consequences of high school 
education bloom several years after students have gradu- 
ated, it is important that states follow the progress of stu- 
dents as they move through their education. An effective 
longitudinal data system would include elements such as 
high school grades and assessment results, college course- 
taking patterns, success in first-year college courses, and 
persistence and completion rates. It goes without saying 
that such a system should be able to capture these data in 
a manner that protects student privacy. Such confidenti- 
ality will be critical as states move forward. 



Achieve launched this study to help inform the decisions states are making about high school assessments by providing greater 
insights into the world of college admissions and placement testing. We hope that, armed with better information about the 
nature of these transition tests, states can make better decisions about the kinds of tests to employ to help ensure that all stu- 
dents graduate prepared to succeed beyond high school. 
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Appendix A: Tests Analyzed by Achieve 

The following provides detailed information on each of the tests analyzed for this study. 
With the exception of the ACCUPLACER and COMPASS, Achieve analyzed the tests in 
their entirety. In the case of ACCUPLACER and COMPASS — which are computer- 
adaptive tests — Achieve analyzed a stratified sample of items that represents a large subset 
of the broader pool of items. 



National Admissions Tests 

ACT Assessment. Begun in 1959 as a project of the 
American College Testing Program, the ACT Assessment 
is a curriculum-based achievement test that measures stu- 
dent knowledge and skills in four areas: English, reading, 
mathematics and science reasoning. An optional writing 
test measures skill in writing a short essay. The ACT 
consists of four multiple-choice tests — one in each area 
— with each scored on a scale of one to 36; the compos- 
ite score reported by ACT is the average of the four test 
scores. The ACT is explicitly curriculum-based, designed 
to assess what students have learned in their high school 
courses. 16 Many colleges use ACT scores as surrogates for, 
or supplements to, other placement procedures. For this 
study, Achieve examined two separate forms of the ACT 
tests of English, reading, direct writing and mathematics. 
In addition, Achieve examined 22 items from the ACT 
science test that assess mathematical skills in contextual 
settings. 

SAT Reasoning Test. Sponsored for more than seven 
decades by the College Board in New York City, the SAT 
Reasoning Test is intended to measure critical thinking 
skills required for academic success in college. Originally 
called the Scholastic Aptitude Test, the SAT historically 
consisted of two sections, verbal and mathematical, each 
scored on a scale of 200-800. (Each section is scaled 
so that 500 represents the average score with a standard 
deviation of 100 points.) Due to recent changes, the 
SAT currently consists of three sections: critical read- 
ing, which has sentence completion and passage-based 
questions; writing, which has multiple-choice questions 
and a written essay; and mathematics, which is based on 



“topics from up through a third-year college preparatory 
course.” 17 Each section of the revised three-hour-and-45- 
minute SAT is scored on a scale of 200-800, with two 
writing subscores — one each for the multiple-choice 
and essay parts of the writing sections. Unlike most other 
tests examined in this study, the SAT mathematics test 
includes some items with student-constructed numeri- 
cal answers. (These are encoded in a special grid for 
computer grading.) The College Board suggests that the 
SAT be used primarily for its intended purpose, namely 
admissions, but notes that SAT scores “may be helpful 
for identifying students in need of remedial training.” 18 
Many colleges use SAT scores as criteria for placement 
or to exempt students from tests that are specifically 
designed for placement purposes. In this study, Achieve 
examined two separate forms each of the critical reading, 
writing and mathematics tests. 

National Placement Tests 

ACCUPLACER. A computer-adaptive placement test 
sponsored by the College Board, 19 ACCUPLACER is 
delivered over the Internet and scored immediately. 

There are five assessments offered (reading comprehen- 
sion, sentence skills, arithmetic, elementary algebra 
and college-level mathematics) and an optional essay 
section called WritePlacer. In regard to placement into 
introductory, credit-bearing mathematics courses, the 
institutions Achieve surveyed most commonly reported 
using elementary algebra and college-level mathematics. 
In a computer-adaptive environment, the selection and 
sequencing of questions presented to each student varies 
according to how students answered previous questions. 
ACCUPLACER can be set up to apply an institution’s 
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placement rules immediately and to produce a variety of 
diagnostic reports. It also allows colleges to edit problem 
banks to include institutional-specific questions. In this 
study, Achieve analyzed a large sample of items (stratified 
by domain and difficulty level) from ACCUPLACER’s 
reading comprehension, sentence skills, elementary 
algebra and college-level mathematics tests. On the read- 
ing comprehension and the sentence skills tests, Achieve 
analyzed 90 and 86 items respectively; students answer 
20 items on each of these tests. On the elementary alge- 
bra and college-level mathematics tests, Achieve analyzed 
176 and 125 items respectively; students take 12 items on 
the elementary algebra test and 20 items on the college- 
level mathematics test. Because this is a computer- 
adaptive test, students taking the test answer different 
sets of questions. “The first question presented is of aver- 
age difficulty and is chosen randomly from several starter 
questions of the same level of difficulty. If a student 
answers the question incorrectly, the next question to be 
administered is chosen from a group of easier questions, 
whereas a correct answer will cause the next question to 
be somewhat more difficult.” 20 

COMPASS. Developed by ACT, 21 the COMPASS 
assessments are computer-adaptive placement tests 
designed to assist colleges in placing students into 
appropriate introductory or developmental (remedial) 
courses. The COMPASS tests produce scores in reading, 
writing (usage, mechanics), numerical skills (prealgebra), 
algebra, college algebra, geometry and trigonometry. 

The system also includes an electronically scored direct 
writing assessment called e-Write. Colleges that use 
COMPASS can select which tests are administered, as 
well as their order, length and cut scores. COMPASS is 
available as an onsite system and, recently, for delivery 
over the Internet. In this study, Achieve analyzed a large 
sample of items (stratified by domain and difficulty level) 
from four COMPASS tests: reading, writing, algebra 
and college algebra. Achieve analyzed 1 17 reading test 
items, 238 writing test items, 120 algebra test items and 
100 college algebra test items. Students take varying 
numbers of test items and on average take 12 to 29 
items in reading, 23 to 52 items in writing, eight to 
20 items in algebra and eight to 20 items in college 
algebra. The number of test items fluctuates because 



this is a computer- adaptive test with varying numbers 
of questions depending on the difficulty of the test. 
COMPASS draws from a pool of 2,400 test items, and 
the computer-adaptive format “adjusts the difficulty level 
to the skills of the individual student, eliminating items 
that are too easy or too difficult and that contribute little 
to the measurement.” 22 

State or Systemwide Tests 

California State University. The Early Assessment Pro- 
gram (EAP) of California State University (CSU) is used 
as one of many placement tests in determining course 
placement. The EAP test is administered to 11th grade 
students in California by the California Department 
of Education for use at CSU in determining whether a 
student needs a course in remedial reading. Students also 
may avoid remedial reading with a score of 550 or higher 
on the SAT I verbal section, a score of 680 or above on 
the SAT II Writing Test, a score of 24 or higher on the 
ACT Assessment English test, a score of three or higher 
on the Advanced Placement Language and Composi- 
tion or Literature and Composition tests, or previous 
college-bearing credit (with a grade of C or higher) in a 
course that satisfies the requirement in English composi- 
tion. Achieve reviewed the writing portion of the EAP. It 
consists of a 45-minute essay on a topic given at the time 
of the test. Students are asked to explain the author’s 
argument and discuss the extent to which they agree or 
disagree with the author’s views. 

Texas Higher Education Assessment. Since 1989, 
the Texas Higher Education Assessment (THEA) has 
provided Texas institutions of higher education with 
tests of reading, writing and mathematics designed to 
assess skills that entering freshman-level students should 
have if they are to perform effectively in undergraduate 
certificate or degree programs in Texas public colleges or 
universities. The test is offered in three forms: a paper- 
and-pencil test (which Achieve analyzed), a computer- 
administered test, and a “quick test” administered by 
colleges and universities to their entering students. The 
THEA writing test includes a brief writing sample. Test 
sessions last five hours, during which students can work 
on as few as one and as many as three sections of the test. 
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Mathematics Placement Test (Washington) . Admin- 
istered since 1984, Washington state’s Mathematics 
Placement Test was developed by mathematics faculty at 
Washington’s public baccalaureate institutions to assist 
students and their academic advisers in selecting first- 
year mathematics courses. The test is available at two 
levels. The Intermediate Math Placement Test, intended 
for students who have fewer than three or four years of 
high school mathematics, consists of 35 items covering 
elementary and intermediate algebra and the first part of 
Precalculus. The Advanced Math Placement Test is 
intended for students who have taken at least three or 
four years of high school mathematics and covers inter- 
mediate algebra and a full year of precalculus. 

Institution-Level Tests 

Louisiana State University. Achieve considered two 
placement tests given by Louisiana State University 
(LSU) — one in writing, the other in mathematics. 
Incoming freshmen at LSU may challenge their place- 
ment into an entry-level English course — determined 
on the basis of ACT, SAT or Advanced Placement scores 

— by taking part in LSU’s On-Line Challenge Program, 
a 1,200-1,500 word essay submitted via the Internet. 
Students are encouraged to write multiple drafts before 
sending in the final essay, which they have three days 

to complete. The College Algebra Placement/Credit 
Exam is an optional test consisting of 25 multiple-choice 
questions. Initial placement in LSU mathematics courses 
is based on the mathematics section of the ACT or the 
SAT. A sufficient score on the ACT or SAT allows first- 
year students into College Algebra. However, students 
have the option of taking the LSU College Algebra 
Placement/ Credit Exam either to improve their initial 
placement or to earn credit for and placement out of 
College Algebra. By validating credit for College Algebra 

— the first credit-bearing course — rather than merely 
verifying readiness for this course, the LSU test is more 
advanced than most others in this study. 

Mathematical Association of America Placement 
Tests. Beginning in 1977, the Mathematical Association 
of America (MAA) developed a suite of placement 
tests intended to assist mathematics departments in 



developing effective placement procedures. These 
tests covered arithmetic and basic skills, basic algebra, 
algebra, advanced algebra, trigonometry and elementary 
functions, and calculus readiness; most were offered in 
both calculator and non-calculator versions. Recently, 
MAA contracted with MapleSoft to develop computer 
versions of these tests. In this study, Achieve analyzed 
the paper-and-pencil, non-calculator versions of MAA’s 
algebra and calculus readiness tests. The MAA test is 
categorized as an “institution test” because it is used by 
individual institutions (rather than by systems) with the 
option of customizing the test to meet the needs of their 
programs. 

University of Maryland-Baltimore County. Entering 
first-year students at the University of Maryland- 
Baltimore County must write an online English Compo- 
sition Placement Essay. The scores on the test determine 
a student’s placement in an English composition course. 
The test can be administered to a student only once, and 
students are not allowed to appeal the results of the score 
of the essay or the course in which the student is placed. 23 
Students have 40 minutes to complete the essay, which 
consists of writing on a topic given at the time of the test. 

University of Maryland-College Park. During orienta- 
tion at the University of Maryland-College Park, each 
new undergraduate takes the Mathematics Placement 
Test, which covers arithmetic, as well as content typi- 
cally found in courses such as Algebra I, Algebra II and 
Trigonometry. The results are used to advise students on 
the appropriate mathematics course in which to enroll. 
Students who score poorly may take a non-credit survey 
of Algebra I and Algebra II for a special fee. 24 Students 
who fail to place into the course they desire are allowed 
to retake the placement test once during each semester. 

University of Massachusetts-Amherst. Students at 
the University of Massachusetts-Amherst may either 
enroll in an entry-level writing course or take the Writ- 
ing Placement Exam of the school’s Writing Program to 
place into a higher-level course. The 60-minute exam 
consists of an essay in which the student is asked to write 
on one of two topics given. Students are graded by eight 
Writing Program teachers, and the essay is graded on a 
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student’s ability to demonstrate understanding of the pas- 
sage, organize and focus on a thesis, develop the thesis, 
and exhibit an understanding of correct grammar and 
sentence structure. 25 

University of Minnesota. To help determine their 
placement in mathematics, most students (except those 
who have a previous college credit in Calculus) entering 
the University of Minnesota take one of two mathematics 
placement tests: the College Math Readiness Test or the 
Calculus Readiness Test. Which test a student takes is 
determined by the student’s ACT score or, in some cases, 
the courses he or she took in high school. Each test is 60 
minutes and consists of 30 multiple-choice item; calcula- 
tors are not permitted on the test. Students take the test 
via the Internet. Although results on the tests do not 
guarantee placement in a specific course, they are used to 
help students and their advisers determine the appropriate 
level of first-year placement. 

The College of New Jersey. An incoming student at 
The College of New Jersey who does not meet a thresh- 
old score on the SAT, the Advanced Placement English 
or history tests, or the college’s Writing Exemption and 
Placement Exam is required to take an entry-level writing 
course. On the Writing Exemption and Placement Exam, 
students are scored on a six-point rubric and are placed 
accordingly. 26 The exam consists of an essay written on a 
word processor at a campus computer lab. The topic for 
the essay comes from a newspaper article, and students 
have 60 minutes to read the article, plan a response and 
write the essay. 

Purdue University-North Central. The Student 
Assessment and Measurement system at Purdue’s North 
Central campus is a series of three tests in English, math- 
ematics and reading that places a sizable subset of incom- 
ing students into entry-level courses. 27 All students who 
graduated in the lower half of their high school class or 
have a GED are required to take all three tests; others are 
required to take specific tests based on a review of their 
high school record. For this study, Achieve looked at the 



Form D mathematics test and the English Composition 
Placement Essay. The mathematics test is taken prima- 
rily by students who had less than three years of academic 
mathematics in high school or have been out of high 
school for several years. Their score places them into one 
of two lower-level algebra classes, neither of which counts 
toward college mathematics credit. The English Compo- 
sition Placement Essay is a 45-minute writing assignment 
that consists of at least two single-spaced pages on an 
assigned topic given to students at the time of the test. 

On the basis of these results, students are placed into 
one of two lower-level English composition courses. 

Temple University. Prior to their first semester of study, 
all entering students at Temple University in Philadelphia 
must take placement tests in English and mathematics. 
The English Placement Test is required for all fresh- 
men and for transfer students who are not transferring 
a credit-bearing, college-level writing course. The test 
consists of an essay component (which Achieve analyzed) 
and a multiple-choice section that tests skills related to 
sentence structure, conventions of written English and 
reading comprehension. For the essay, students are asked 
to analyze an author’s position on an issue and offer their 
own opinion. Scores on each section are used to aid in 
placement decisions. The Mathematics Placement Test 
is a three-part test consisting of 50 multiple-choice items 
beginning with elementary computation (whole num- 
bers, fractions and ratios), progressing through algebra 
and concluding with calculus readiness. Placement 
depends on achievement levels on each part of the exam. 
The test analyzed for this study devotes the majority of 
questions (85 percent) in approximately equal parts to 
elementary computation and algebra. A revised test, 
to be introduced next year, will reduce the proportion 
of elementary questions and increase the proportion 
devoted to algebra. 

Youngstown State University. Until recently, 
Youngstown State University (YSU) used a writing 
Composition Placement Test for incoming freshmen to 
determine placement in English courses. In the last year, 
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YSU switched to using the COMPASS Reading test. 28 
Achieve reviewed the Composition Placement Test prior 
to this switch. Incoming students who did not meet a 
threshold on the ACT or SAT were preplaced into 
entry-level English courses. Such students could either 
accept their course placement or attempt to place into a 



different course by taking the Composition Placement 
Test. The exam consisted of a 40-minute handwritten 
essay. During the exam, students were presented with a 
topic and then wrote an essay where they took a stance 
on the topic issue and demonstrated their understanding 
of the ideas expressed in the topic. 



OVERVIEW OF MATHEMATICS TESTS ANALYZED BY ACHIEVE (in alphabetical order) 



Name of Test 


Form 


Version 

Date 


Multiple 

Choice 


Constructed 

Response 


Total 

Items 


Calculator 


Timing 

(Minutes) 


Type 


ACCUPLACER — 

Elementary 

Algebra 


N/A 


N/A 


176 


0 


176* 


Depends 


None 


National 

Placement 


A computer-adaptive test; the test is undated. Student may use calculator when question allows it. 
*Represents number of test items reviewed. Students taking ACCUPLACER answer 12 test questions. 


ACCUPLACER — 
College Level 
Mathematics 


N/A 


N/A 


125 


0 


125* 


Depends 


None 


National 

Placement 


A computer-adaptive test; the test is undated. Student may use calculator when question allows it. 
*Represents number of test items reviewed. Students taking ACCUPLACER answer 20 test questions. 


ACT 

Assessment — 
Math 


60E/ 

61D 


2004- 

OS 


60 


0 


60 


Yes 


60 


National 

Admissions 


COMPASS — 
Algebra 


N/A 


N/A 


120 


0 


120* 


Yes 


None 


National 

Placement 




A computer-adaptive test. 

*Represents number of test items reviewed. Students taking COMPASS answer varying numbers of test 
questions averaging eight to 20 items, depending on a wide variety of factors. 


COMPASS — 
College Algebra 


N/A 


N/A 


100 


0 


100* 


Yes 


None 


National 

Placement 


A computer-adaptive test. 

*Represents number of test items reviewed. Students taking COMPASS answer varying numbers of test 
questions averaging eight to 20 items, depending on a wide variety of factors. 


Louisiana State 
University 
College Algebra 
Placement/ 
Credit Exam 


N/A 


2002 


25 


0 


25 


No 


None 


Institution 

Level 


Mathematical 
Association 
of America 
Algebra 
Placement Test 


4H 


1997 


32 


0 


32 


No* 


40 


Institution 

Level 


*A calculator version also exists. 



(continued on next page) 
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(continued) 



Name of Test 


Form 


Version 

Date 


Multiple 

Choice 


Constructed 

Response 


Total 

Items 


Calculator 


Timing 

(Minutes) 


Type 


Mathematical 
Association 
of America 
Calculus 
Readiness 
Placement Test 


1G 


1994 


25 


0 


25 


No* 


30 


Institution 

Level 


*A calculator version also exists. 


Purdue 
University- 
North Central 
Student 

Assessment and 
Measurement 
Placement Test 


D 


2006 


40 


0 


40 


No 


45 


Institution 

Level 


SAT Reasoning 
Test — Math 


4BSA08/ 2005 44 10 54 Graphing 70 National 

4BSA01 Admissions 




In three sections: two are multiple choice and one is a mixture of multiple choice and constructed response. 


Temple 
University 
Mathematics 
Placement Test 


N/A 


12/2005 


50 


0 


50 


No 


60 


Institution 

Level 


Texas Higher 

Education 

Assessment 


X 2003 53 0 53 4-Function 


300* Statewide 


*Test sessions last five hours; a student may also use this time to complete either or both of the reading 
and writing sections of the test. 


University of 
Maryland- 
College 
Park Math 
Placement Test 


N/A 


2003 


63 


0 


63 


Non- 

graphing 


70 


Institution 

Level 


University of 
Minnesota 
Calculus 
Readiness Test 


N/A 


1995 


30 


0 


30 


No 


60 


Institution 

Level 


University of 
Minnesota 
College Math 
Readiness Test 


N/A 


1995 


30 


0 


30 


No 


60 


Institution 

Level 


Washington 
Intermediate 
Mathematics 
Placement Test 


N/A 


2000 


35 


0 


35 


No 


60 


Statewide 


Washington 
Advanced 
Mathematics 
Placement Test 


N/A 


2000 


30 


0 


30 


No 


60 


Statewide 
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OVERVIEW OF ENGLISH LANGUAGE ARTS TESTS ANALYZED BY ACHIEVE (in alphabetical order) 



Name of Test 


Form 


Version 

Date 


Reading 

Items 


Language 

(Indirect 

Writing) 

Items 


Total 

Items 


Direct 

Writing 


Timing 

(Minutes) 


Type 


ACCUPLACER — 

Reading 

Composition 


N/A 


N/A 


90* 


0 


90 


N/A 


None 


National 

Placement 


A computer-adaptive test. 

*Represents number of test items reviewed. Students taking ACCUPLACER answer 20 test questions. 


ACCUPLACER — 
Sentence Skills 


N/A 


N/A 


0 


86* 


86 


N/A 


None 


National 

Placement 




A computer-adaptive test. 

*Represents number of test items reviewed. Students taking ACCUPLACER answer 20 test questions. 


ACT 

Assessment — 
English and 
Reading 


60E/ 

61D 


2004- 

OS 


40 


75 


115 


N/A 


80 


National 

Admissions 


Seventy-five language questions in 45 minutes and 40 reading questions in 35 minutes. 


ACT 

Assessment — 
Writing 


10G 


2005 


N/A 


N/A 


N/A 


Essay 


30 


National 

Admissions 


California State 
University Early 
Assessment 
Program — 
Direct Writing 


N/A 


2005 


N/A 


N/A 


N/A 


Essay 


45 


Statewide 


The College 
of New Jersey 
Writing 
Exemption and 
Placement Test 


N/A 


2006 


N/A 


N/A 


N/A 


Essay 


60 


Institution 

Level 


COMPASS — 
Reading and 
Writing Skills 


N/A 


N/A 


117* 


238* 


455 


N/A 


None 


National 

Placement 


A computer-adaptive test. 

*Represents number of test items reviewed. Students taking COMPASS take varying numbers of test 
questions. Students average 12 to 29 items in reading and 23 to 52 items in writing, depending on a wide 
variety of factors. 


COMPASS 

e-Write 


N/A 


N/A 


N/A 


N/A 


N/A 


Essay 


None 


National 

Placement 


Louisiana State 
University 
Writing Test 


N/A 


2006 


N/A 


N/A 


N/A 


1,200- to 
1,500- 
word 
essay 


None 


Institution 

Level 


Students read preselected essays and then write on a specified topic. 



(continued on next page) 
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(continued) 



Name of Test 


Form 


Version 

Date 


Reading 

Items 


Language 

(Indirect 

Writing) 

Items 


Total 

Items 


Direct 

Writing 


Timing 

(Minutes) 


Type 


Purdue 

University- 

North Central 

English 

Composition 

Placement 

Essay 


Topics 

A-P 


2006 


N/A 


N/A 


N/A 


Two- 

page, 

singe- 

spaced 

essay 


45 


Institution 

Level 


SAT Reasoning 
Test — Critical 
Reading and 
Writing 


4BSA08/ 

4BSA01 


2005 


67 


49 


116 


Essay 


145 


National 

Admissions 


Three critical reading sections, two at 25 minutes and one at 20 minutes. Two indirect writing sections, one 
with 35 questions and one with 14 questions, both at 25 minutes. One essay section is 25 minutes. 


Temple 

University 


W06-J/ 

W06-K 


12/2005 


N/A 


N/A 


N/A 


350-word 

essays* 


60 


Institution 

Level 


English 
Placement 
Test — Essay 
Component 


*Student chooses one of two essay topics from two sets of topics. Students may prepare notes on the essay 
and bring them on the day of the test. 


Texas Higher 

Education 

Assessment 


X 


2003 


42 


41 


83 


300- to 

600-word 

essay 


300* 


Statewide 


*Test sessions last five hours, during which students may complete one or all sections of the test. The 
writing and reading sections of this test might be completed in separate five-hour sessions. The writing 
section includes multiple-choice writing questions and the essay. 


University of 

Maryland- 

Baltimore 

County English 

Composition 

Placement 

Essay 


N/A 


2003 


N/A 


N/A 


N/A 


Essay 


40 


Institution 

Level 


University of 

Massachusetts- 

Amherst 

Writing 

Placement 

Exam 


N/A 


2006 


N/A 


N/A 


N/A 


Essay 


60 


Institution 

Level 


Students write one essay from two given topics. 


Youngstown 
State University 
Composition 
Placement Test 


N/A 


2006 


N/A 


N/A 


N/A 


Essay 


40 


Institution 

Level 
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Appendix B: Reading Methodology 

To analyze admissions and placement tests in reading, each item was coded according to a 
range of lenses designed to capture different characteristics of individual test items and the 
tests as a whole. To ensure the reliability of the data, at least two experts trained in the use 
of the criteria coded each test. Those experts reconciled any differences in coding before the 
data were analyzed. The following are descriptions of the various criteria. 



Content Expectations of an Item 

To identify the content on admissions and placement 
tests of reading, Achieve augmented the framework 
that it used in its study of state exit exams in 2004 (Do 
Graduation Tests Measure Up?). What follows are descrip- 
tions of the original framework and the augmentations 
applied for this study. 

For the 2004 study of state exit exams, Achieve adapted a 
comprehensive listing of the domain of reading, origi- 
nally developed by the Council of Chief State School 
Officers (CCSSO) in collaboration with several states for 
its Survey of Enacted Curriculum. The list was intended 
to fully encompass all topics addressed in reading classes 
from the primary to the secondary level. Based on this 
framework, Achieve developed a taxonomy that included 
all the aspects of reading described in state standards 
— and therefore targeted on state tests — to describe as 
accurately as possible the content or topic that each item 
measured. The listing used in Achieve’s study was revised 
to more clearly reflect the topics addressed by test items 
at the secondary levels. Because the list was originally 
developed to cover all grades from kindergarten through 
grade 12, some codes that were irrelevant for higher-level 
tests have been deleted. In addition, the listing has been 
reorganized to clarify the relationship of the elements. 

In Achieve’s 2004 study, the major reporting categories 
for reading were as follows: 

■ Basic comprehension (includes word definitions, main 
idea, theme and purpose) 

■ Literary topics (includes figurative language, poetic 
techniques, plot and character) 



■ Informational topics (includes structure, evidence and 
technical elements) 

■ Critical reading (includes appeals to authority, 
reason and emotion; validity and significance of 
assertion or argument; style in relation to purpose; 
and development and application of critical 
criteria) 

For the current study of admissions and placement tests, 
Achieve regrouped some of the codes into more discrete 
categories to streamline reporting. For example, argu- 
ment and assertion, formerly under critical reading, are 
both aspects of persuasive texts and are now grouped 
under informational/persuasive elements. Codes also 
were realigned into groupings that reflect all of the ele- 
ments within them. For example, all the literary elements 
have been grouped together — narrative elements with 
the author’s craft elements. Additionally, some elements 
formerly included in the critical thinking category were 
deleted because they combined references to both content 
and cognitive demand, such as determining the valid- 
ity of an assertion. Redundancies in the list also were 
eliminated. 

This revision yields four revised major categories of codes 
for reading: 

■ Vocabulary (includes word definitions) 

■ General comprehension (includes purpose and 
main idea) 

■ Literary elements (includes figurative language, plot 
and character, theme, setting, and poetic language) 
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■ Informational/persuasive elements (includes organiza- 
tion and structure, assertions, evidence, and technical 
elements) 

Cognitive Demand 

In its 2004 study of state exit exams, Achieve used a 
taxonomy of performance expectations derived from 
CCSSO’s description of performances in its Survey of 
Enacted Curriculum and influenced by Achieves 
assessments-to-standards alignment protocol. This 
taxonomy was categorized into four levels of cognitive 
demand of reading items. The four levels provided infor- 
mation on the kind and complexity of reasoning required 
of students, ranging from simple recall of information to 
complex reasoning skills. The former categories were as 
follows: 

1 . Literal recall 

2 . Infer 

3 . Explain 

4 . Analyze 

Some revisions have been made to the former categories, 
using a revised taxonomy based on Bloom that was devel- 
oped by Anderson and Krathwohl in 2001. 29 The revised 
taxonomy retains much of Bloom’s 1956 model. How- 
ever, the original Bloom taxonomy combined content 
and performance while the revised taxonomy separates 
the content from the performance. Several of the original 
codes in the former Achieve cognitive demand scale also 
included content topics as well as processes, making 
the codes for some items redundant, such as “identify- 
ing main ideas or theme,” where both main idea and 
theme are now coded with an appropriate content code. 
Additionally, the revised taxonomy differentiates low- 
level inference (called interpreting) from more complex 
inference (called inferring). Thus, the revised taxonomy 
appears to allow a cleaner, more specific categorization of 
test items’ cognitive demands. 

The revised scale retains some of the same headings as 
the original, with some expansion that allows for a better 
discrimination among cognitive processes typically 
assessed in reading tests: 



1 . Recall (includes locating and recognizing) 

2 . Low inference (includes paraphrasing and 
generalizing) 

3 . High inference (includes concluding, comparing and 
illustrating) 

4 . Analysis (includes discriminating and outlining) 

5 . Evaluating (includes critiquing) 

6 . Creating (includes designing and hypothesizing) 

Categories five (evaluating) and six (creating) are not 
characteristic of items found on large-scale, on-demand 
state tests and are therefore not included in the related 
data charts. 

Demand of Reading Passages 

Achieve analyzed the difficulty level of each reading pas- 
sage according to a six-point scale ranging from straight- 
forward text to more complex, challenging and abstract 
text. To develop the scale, reading experts examined 
a large collection of reading passages that appear on 
tests students take in the transition from high school to 
college. Using characteristics such as specialization of 
vocabulary, predictability of text organization, complex- 
ity of syntax, level of abstractness, familiarity of the topic 
and number of concepts introduced in the passage, they 
defined six levels of reading demand. Generally speaking, 
Level 1 represents upper-elementary reading levels, Levels 
2 and 3 represent middle school-level reading, Level 4 
represents early-stage high school reading, and Levels 5 
and 6 represent late-stage high school reading. 

Elements for consideration of reading passage difficulty 
include: 

■ Structure 

• Narration 

• Description 

• Explanation 

• Instruction 

• Argumentation 
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■ Vocabulary 

• Poetic 

• Idiomatic 

• Technical 

• Unusual/unfamiliar 

■ Syntax/ connectives 

• Dialogue 

• Sentence structure 

■ Characters/ideas 

■ Narrator/stance 

■ Theme/message/moral 

■ Literary effects 

• Foreshadowing 

• Flashback 

• Irony 

■ Familiarity 

• Topic 

• Place 

• Time period 

Reading Rigor Index 

The difficulty of an item on a reading test is determined 
both by the cognitive demand of the question and by 
the complexity of the reading passages the question 
addresses. To capture this important interplay, Achieve 
developed a Reading Rigor Index (RRI) that combines 
the cognitive level of a question with the difficulty 
level of the passage to which the question refers. Specifi- 
cally, the RRI of a reading item is defined to be the sum 
of the cognitive demand and the reading demand levels 
of the item. Since the cognitive demand scale ranges 
from one to five, and the reading passage demand scale 
from one to six, the RRI has 10 levels ranging from 
two to 11. The low end of the scale reflects less demand- 
ing questions about less complex reading passages; 
the high end reflects challenging questions about 
complex texts. 



Genre 

In its 2004 study of state exit exams, Achieve coded the 
reading passages into three categories: narrative, informa- 
tional and visual media. For the purposes of this study, 
the consideration of genre was informed by the National 
Assessment of Educational Progress 2009 Reading Frame- 
work that recommends two types of texts be included 
on the assessment: literary texts, which include fiction, 
literary non-fiction and poetry; and informational texts, 
which include exposition, argumentation and persuasive 
text, and procedural text and documents. These genres 
were coded into seven categories as shown below. 

■ Literary text 

• Short stories /novel excerpts 

• Poetry 

• Drama 

■ Literary non-fiction 

• Essays 

• Autobiographies/biographies 

• Literary speeches 

■ Exposition 

• News stories 

• Textbooks /informational articles 

■ Argumentation/persuasive 

• Editorials 

• Political speeches 

• Persuasive essays 

■ Procedural text/documents 

• Manuals 

• Directions 

■ Media 

• Photographs 

• Advertisements 

■ Graphics 

• Tables 

• Maps 
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Appendix C: Writing Methodology 

To analyze admissions and placement tests in writing, Achieve reviewed two types of ques- 
tions: multiple-choice questions, which are indirect measures of writing ability, and writing 
prompts and supporting materials, which are direct measures of writing ability. 



Indirect Writing Methodology 

Multiple-choice questions (indirect measures of writing 
ability) were coded for three elements: content, cognitive 
demand and level of challenge. 

Content 

To analyze the content focus of indirect writing items in 
its 2004 study of state exit exams (Do Graduation Tests 
Measure Up?), Achieve adapted the content descriptions 
originally developed by the Council of Chief State School 
Officers (CCSSO). The resulting taxonomy was divided 
into three categories: 

■ Writing process 

■ Writing applications 

■ Language study 

In the current study, it was necessary to revisit the cat- 
egories to ensure that the items are clearly described in 
terms of their content targets. A major revision to content 
categories was in regard to the writing process category, 
which was originally conceived as describing what is 
taught in this area, such as prewriting, rather than what is 
typically tested, such as editing skills. This category was 
revised to more accurately capture the elements that are 
assessed commonly on high school and college exams; it 
was renamed “Strategies, Organization and Style” to bet- 
ter represent the elements now included in the category. 
The former language study category also was renamed 
as “Grammar, Usage, Mechanics” to more accurately 
describe the items coded to it. The writing applications 
category was not used in the analysis of multiple-choice 
items; therefore, it was deleted from the set. (Writing 
applications /direct writing assessments were evaluated 
holistically. See next section.) 



The revised content categories for indirect writing items 
are as follows: 

■ Strategies, Organization and Style 

• Transitions and topic sentences 

• Adding/deleting details and evidence 

• Purpose — whole text 

• Sentence reorganization 

• Paragraph reorganization 

• Passage reorganization 

• Redundancies/wordiness 

• Word choice, tone 

■ Grammar, Usage, Mechanics 

• Spelling 

• Mechanics (capitalization, italicizing, underlining) 

• Punctuation (commas, apostrophes, semicolons, 
parentheses, dashes, end punctuation) 

• Sentence structure (connecting and transition 
words, subordinate/dependent clauses, sentence 
fragments, comma splices, run-on sentences, mis- 
placed modifiers, parallelism) 

• Grammatical analysis (identification of word part, 
naming of word part) 

• Language usage (subject/verb agreement, pronoun/ 
antecedent agreement, pronoun case, verb tense, 
adjective/adverb use, idioms, comparative/superla- 
tive modifiers) 

Cognitive Demand 

The scale for cognitive demand for indirect writing items 
is the same as that used for analyzing reading items. For 
example, Level 1 (recall) includes the skill of recognizing 
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or identifying, matching knowledge in long-term 
memory that is consistent with presented material 
(e.g., identify literary terms, correct sentence errors). 

See page 49 for a full description. 

Level of Challenge 

Achieve adapted the ACT Standards for Transition for 
Writing, which are based on evidence derived from 
student responses to indirect writing items. These 
descriptions of skills refer not only to the specific skill 
but also to its use in a simple-to-complex context. 
Divided into seven levels, these standards describe skills 
in such areas as topic development, organization, word 
choice, sentence structure, conventions of usage and 
punctuation. At each level within these categories are 
examples of skills at varying levels of complexity. 

The categories include: 

■ Topic development in terms of purpose and focus 

■ Organization, unity and coherence 

■ Word choice in terms of style, tone, clarity and 
economy 

■ Sentence structure and formation 

■ Conventions of usage 

■ Conventions of punctuation 

■ Grammatical analysis 

See page 52 for examples of the three most advanced 
levels of three of the above categories from the ACT Stan- 
dards for Transition for Writing. (For a full description, 
see www.act.org/standard/planact/english/index.html.) 

Direct Writing Methodology 

Direct Writing Assessment 

The majority of admissions and placement tests of writ- 
ing analyzed by Achieve measure writing directly by 
asking students to produce an essay. Unlike the reading 



and indirect writing multiple-choice items that can be 
considered individually and the data aggregated to reveal 
trends, samples of student writing are given one score 
based on the scoring guides developed by each testing 
vendor or institution. Although an indirect writing item 
may focus on one independent skill such as word choice, 
the many and various parts that contribute to a piece of 
writing are considered interdependently. 

The vast majority of direct writing assessments are com- 
posed of three parts: 

■ The writing prompt that sets the task for the writer 
and often describes aspects of the topic. In some cases 
a text or quote is provided for student response. 

■ Scoring guides or rubrics that describe the score levels, 
usually either four or six levels, listing for each level a 
description of the elements that characterize writing at 
that level. 

■ Anchor or sample papers that are examples of student 
responses to the topic at a variety of score levels. 

For all of the direct writing assessments reviewed by 
Achieve, each of these elements was collected (as avail- 
able) and examined. The prompts were characterized 
in terms of the type of writing they intended to elicit, 
such as personal narrative, defense of a point of view or 
description. The scoring guides then were reviewed to 
determine the kinds of criteria being examined by the 
raters of the assessments, with particular attention to 
the inclusion of reading criteria when such texts were 
provided as part of the writing task, and the inclusion of 
elements of argumentation when this type was specified 
in the writing task. Sample papers and commentary on 
them also were reviewed as an aid to exemplifying the 
elements in the respective scoring guides. Finally, the 
direct writing assessments were examined with attention 
to those skills most necessary for success in the college 
classroom as delineated in the ADP English benchmarks. 
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Sentence Structure and Formation 


Level 4 


Maintain consistent verb tense and pronoun person on the basis of the preceding clause or 
sentence. 


Level 5 


Maintain a consistent and logical use of verb tense and pronoun person on the basis of 
information in the paragraph or essay as a whole. 


Level 6 


Work comfortably with long sentences and complex clausal relationships within sentences, 
avoiding weak conjunctions between independent clauses and maintaining parallel 
structure between clauses. 


Organization, Unity and Coherence 


Level 4 


Add a sentence to introduce or conclude the essay or to provide a transition between 
paragraphs when the essay is fairly straightforward. 


Level 5 


Add a sentence to introduce or conclude a fairly complex paragraph. 


Level 6 


Consider the need for introductory sentences or transitions, basing decisions on a thorough 
understanding of both the logic and rhetorical effect of the paragraph and essay. 


Conventions of Usage 


Level 4 


Ensure that a pronoun agrees with its antecedent when the two occur in separate clauses or 
sentences. 


Level 5 


Ensure that a verb agrees with its subject in unusual situations (e.g., when the subject-verb 
order is inverted or when the subject is an indefinite pronoun). 


Level 6 


Provide idiomatically and contextually appropriate prepositions following verbs in situations 
involving sophisticated language or ideas. 
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Appendix D: Mathematics Methodology 

To analyze admissions and placement tests in mathematics, each item was coded accord- 
ing to a range of lenses designed to capture different characteristics of individual test items 
and the tests as a whole. Some of the criteria in English language arts and mathematics are 
similar, although there are important differences that stem from the distinct natures of the 
disciplines. To ensure the reliability of the data, at least two experts trained in the use of the 
criteria coded each test. Those experts reconciled any differences in coding before the data 
were analyzed. The following are summaries of the various criteria. 



Content Expectations of the Item 

This lens compares the content of admissions and place- 
ment tests, using the Third International Mathematics 
and Science Study (TIMSS) Mathematics Framework 
adapted for use in this study by the U.S. TIMSS 
National Research Center at Michigan State University 
and Achieve experts. The framework provides a detailed, 
comprehensive taxonomy of mathematics content, orga- 
nized at its most general levels according to the following 
major domains of mathematics: 

■ Number 

■ Algebra 

■ Geometry/measurement 

■ Data 

These domains are further broken down into smaller 
units to allow for finer-grained comparisons. For 
example, geometry content is divided into a variety 
of categories such as two-dimensional geometry and 
measurement; three-dimensional geometry and mea- 
surement; transformations, congruence and similarity; 
and trigonometry. These categories are subdivided even 
further to facilitate a high degree of content specificity in 
coding. Item coders for this study assigned up to three 
primary content codes — and sometimes secondary 
content codes — to each test item. In many cases, the 
multiple content codes aligned with the same reporting 
category (e.g., geometry/measurement or algebra), but 
this was not always the case. Items that aligned with 



more than one reporting category were re-examined, and 
one primary code was identified for reporting purposes. 

The content codes included in the original TIMSS 
Mathematics Framework (used by Achieve for its study 
of state exit exams, Do Graduation Tests Measure Up?, 
in 2004) address content that ranges from that typically 
taught at the elementary school level to that taught at 
high school and even beyond. However, the taxonomy 
of content codes from the original TIMSS Mathematics 
Framework is most specific and detailed at the elemen- 
tary, middle school and early high school levels. There- 
fore, for the purposes of this study, staff from the TIMSS 
National Research Center worked with Achieve staff and 
the trained coders to revise and refine the taxonomy to 
include content codes that would describe content in the 
full range of items being seen in the admission and place- 
ment exams included in this study, particularly content 
typically taught at the Algebra II level and beyond. For 
example, the original TIMSS Mathematics Framework 
lacked detail with respect to matrices, the variety of func- 
tions typically encountered in Algebra II and above 
(such as logarithmic, exponential and trigonometric 
functions), and more advanced trigonometry and ana- 
lytic geometry (including polar coordinates and their 
graphs). Although not all tests analyzed for this study 
included such advanced content, it was essential to be 
able to identify and track it when it did occur. There- 
fore, codes corresponding to such content were assigned 
and reporting categories associated with each code were 
defined. 
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Approximate Grade-Level 
Demand of Each Item 

This lens provides a scale to rate the approximate grade 
level of the mathematics content included in tests 
according to the TIMSS International Grade Placement 
(IGP) index. This index was developed by the TIMSS 
National Research Center at Michigan State University 
to facilitate comparison of what is expected of students 
in the United States with what is expected of students in 
other countries. 

The IGP index represents an average, or composite, 
among the 40 countries (other than the United States) 
that participated in the TIMSS and includes both 
high-performing and low-performing countries. An IGP 
is available for each content code within the original 
TIMSS Mathematics Framework and represents the aver- 
age grade level in which a mathematics topic is focused 
on — the point at which the highest concentration of 
instruction on a topic occurs. 

Using their nation’s content standards document, educa- 
tion ministry officials and curriculum specialists in each 
TIMSS country identified the grade level at which a 
math topic is introduced into the curriculum, focused on 
and completed. The IGP index is a weighted average of 
those determinations. 

For those content codes that were added for purposes of 
this analysis, staff from the TIMSS National Research 
Center used their expertise in curricula from the various 
countries to approximate an average IGP index for each. 
An IGP index of 10.8, for example, should be understood 
to mean that the associated content topic is typically 
covered internationally toward the end of 10th grade. An 
average across all the IGP values associated with all of 
the items on a test gives a proxy rating for the difficulty 
of the content encountered on the test. For items that 
span more than one content category and were subse- 
quently assigned a single code for reporting purposes, the 
retained content code tended to be that with the highest 
IGP value. 



The following are examples of the IGP ratings of various 
mathematics topics. 



Content Description 


IGP Index 


Concept of absolute value 


6.8 


Operations with real 
numbers and absolute values 


8.8 


Complex numbers 


11.8 


Inequalities and their 
graphical representations 


9.8 


Relations and their 
properties 


6.8 



Cognitive Demand of Each Item 

This lens provides a taxonomy for what students are 
expected to “do” with the mathematics content in an 
item. As was the case with content, a detailed catalog of 
performance codes was defined by the TIMSS National 
Research Center. For purposes of this study, the cod- 
ers found the range of codes defined in the TIMSS 
Mathematics Framework adequately comprehensive, so 
no additional codes needed to be added. In the process 
of coding test items, coders often found it necessary to 
assign multiple performance codes to an item since stu- 
dents are often called upon to employ multiple skills in 
solving a problem — ranging from recalling basic facts to 
performing computations to developing generalizations. 

For reporting purposes, the performance codes from the 
TIMSS Framework were collapsed into five categories 
that are somewhat hierarchical. The reporting schema 
devised for this analysis was based on a synthesis of the 
TIMSS Mathematics Framework and Achieve’s 
assessment-to-standards alignment protocol used in its 
standards alignment work. Sometimes the multiple per- 
formance codes assigned to an item spanned two or more 
reporting levels, and in those cases, the item was classi- 
fied into the highest corresponding reporting category 
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to most accurately reflect the level of cognitive demand 
required of students to answer an item correctly. The 
reporting levels used to report the cognitive demand of 
items and for computing the average cognitive demand of 
tests are as follows: 

■ Level 1 includes demonstrating basic knowledge or 
recall of a fact or property. 

■ Level 2 includes routine problem solving that asks stu- 
dents to do such things as compute, graph, measure or 
apply a mathematical transformation. 

■ Level 3 includes estimating, comparing, classifying 
and using data to answer a question, or requiring 
students to make decisions that go beyond a routine 
problem-solving activity. 

■ Level 4 includes asking students to formulate a prob- 
lem or to strategize or critique a solution method. 

■ Level 5 includes asking students to develop algo- 
rithms, generalizations, conjectures, justifications or 
proofs. 



Appendix E: ACT and College Board 
Descriptions of Tests 

Following are descriptions provided by administrators of the national admissions and place- 
ment tests reviewed by Achieve in this study. 



ACT 

The ACT Assessment is a timed instrument, with four 
discrete multiple-choice components and an optional 
direct writing component. It was designed to measure 
secondary instruction and college readiness and, as such, 
may be used for admissions and placement decisions. As 
an admissions instrument, the ACT Assessment focuses 
on broader domain decisions. COMPASS is an untimed, 
computer- adaptive program, with content components 
that mirror the ACT Assessment, but configurable fea- 
tures that target postsecondary requirements for efficient 
and effective college placement. As a placement instru- 
ment, the COMPASS focus, by definition, is on narrow 
domain decisions. 

ACT Assessment 

The ACT® Assessment Program (AAP) is a comprehen- 
sive system of data collection, processing and reporting 
designed to assist high school students and postsecondary 
institutions in educational planning and decisionmaking. 
The ACT Assessment is one component of the AAP and 
was developed to assess high school students’ educational 
development and their ability to complete college-level 
work. The ACT Assessment tests of educational develop- 
ment include multiple-choice tests that cover content 
knowledge within four skill areas: English, mathematics, 
reading and science. The optional Writing Test is a 30- 
minute essay test that measures students’ writing skills 
in English. These tests are designed to measure skills 
that are acquired in secondary programs and that are 
most important for success in postsecondary education 
programs. ACT Assessment data are used by high schools 
in academic advising and counseling and by colleges for 
recruitment, admissions and course placement. 



The ACT Assessment tests of educational development 
emphasize reasoning, analysis, problem solving and the 
integration of learning from various sources, as well as 
the application of these proficiencies to the kinds of tasks 
college students are expected to perform. The following 
is excerpted from the ACT Assessment Technical Manual 

(1997): 

Underlying the ACT Assessment tests of educational 
development is the belief that students’ preparation for 
college is best assessed by measuring, as directly as pos- 
sible, the academic skills that they will need to perform 
college-level work. The required academic skills can 
be most directly assessed by reproducing as faithfully as 
possible the complexity of college-level work. Therefore, 
the tests of educational development are designed to 
determine how skillfully students solve problems, grasp 
implied meanings, draw inferences, evaluate ideas and 
make judgments in subject-matter areas important to 
success in college. 

One of the primary purposes of the ACT Assessment is 
to assist in college admission decisions. However, given 
ongoing research and development related to aligning 
ACT Assessment content with secondary and postsec- 
ondary curricula and standards, the tests of educational 
development are also used by postsecondary institutions 
for course placement decisions and by secondary schools 
for program and curricula evaluation. The program- 
and curricula-evaluation use of the ACT Assessment is 
emerging as an important element in analyzing and 
addressing course rigor at the secondary level to ensure 
that students are appropriately prepared for postsecond- 
ary work. The ACT Assessment tests of educational 
development focus on the complexity of college-level 
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courses. This can be contrasted with other instruments 
that focus primarily on academic aptitude. 

COMPASS 

The ACT Computer-Adaptive Placement Assessment 
and Support System (COMPASS™) is a comprehensive 
system that assists postsecondary institutions in plac- 
ing students into appropriate courses. COMPASS offers 
placement and diagnostic tests in mathematics, reading 
and writing. COMPASS e-Write provides direct writ- 
ing assessments used to evaluate student writing. The 
COMPASS program also includes placement testing for 
English as a Second Language (ESL) students. The ESL 
components include ESL Reading, ESL Grammar/Usage, 
ESL Listening and ESL e-Write — a direct writing assess- 
ment specifically designed for ESL students. COMPASS/ 
ESL tests are untimed and computer adaptive. 

The COMPASS program emerged as an outgrowth of 
the ACT Assessment to meet postsecondary institution 
needs specifically relating to course placement. While the 
ACT Assessment is widely used by four-year institutions, 
two-year institutions with open-enrollment options and 
highly fluid student populations have particular need for 
assistance with course placement. These postsecondary 
institutions want to focus testing as efficiently as possible 
to target specific disciplines and accommodate place- 
ment testing in a short period of time. COMPASS was 
developed to meet these needs — a computer-adaptive 
program that allows for testing specific subjects or sets of 
subjects based on institution-specific requirements and 
configurations. 

The COMPASS program allows customization of the 
tests or sets of tests given (e.g., routing across subject 
areas, routing within subject areas, routing from place- 
ment tests to diagnostic tests). The COMPASS features 
can be used to test multiple subjects, including five 
discrete mathematics domains; set institution-specific cut 
scores; produce institution-specific placement scores and 
messages; and provide additional diagnostic testing if 
necessary. The COMPASS program was specifically 
developed to meet a myriad of postsecondary course 



placement needs; however, the COMPASS test content 
was developed to mirror the content articulated within 
the ACT Assessment. The COMPASS mathematics 
domains (Numerical Skills/Prealgebra, Algebra, College 
Algebra, Geometry and Trigonometry) are identical to 
the ACT Assessment math domains. The COMPASS 
Writing Skills test includes items that are comparable 
in content to the ACT Assessment English component, 
although the computer-based format is specifically 
designed as an editing task. The COMPASS Read- 
ing test includes the same passage types (Prose Fiction, 
Humanities, Social Sciences and Natural Sciences) as 
found within the ACT Assessment reading component. 
The only COMPASS difference is that, at the request of 
client institutions, a fifth reading passage type (Practical 
Reading) was added. 

In terms of course placement decisions, the COMPASS 
item pools align with all levels of postsecondary courses. 
For example the COMPASS Numerical Skills/Prealgebra 
pool aligns with Basic College Mathematics and Elemen- 
tary Algebra courses; the COMPASS Algebra pool aligns 
with Elementary Algebra and Intermediate Algebra 
courses; and the COMPASS College Algebra pool aligns 
with Intermediate Algebra, College Algebra and Begin- 
ning Calculus courses. The COMPASS Writing Skills 
pool aligns with Developmental Writing, Freshman 
Composition and College Composition. The COMPASS 
Reading pool accommodates interdisciplinary read- 
ing requirements (i.e., reading across the curriculum) 
and aligns with reading courses such as Developmental 
Reading, Intermediate Reading, Reading in Literature, 
Reading in History and Critical Reading. 

Find Out More 

The ACT Web site includes comprehensive research 
reports (www.act.org/research/reports/) dating from 
1965 to the present related to numerous instruments, 
including the ACT Assessment and COMPASS. The 
ACT research reports are scholarly reports intended to 
inform the community of educational testing profession- 
als. The ACT Web site also includes information briefs 
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(www.act.org/research/briefs/index.html) intended to 
inform general audiences about educational issues high- 
lighted through ACT research. In addition to the ACT 
research reports and information briefs, the ACT 
Web site includes detailed information on the ACT 
National Curriculum Survey® (www.act.org/research/ 
curricsurvey.html). The ACT National Curriculum 
Survey comprises a comprehensive review of state 
educational standards documents, survey of educators 
and consultation with content area experts across the 
curriculum. ACT invites readers to access ACT research 
reports, information briefs and curriculum survey specif- 
ics at the above links to obtain validity evidence for 
admission and course placement uses and in-depth infor- 
mation regarding ACT instruments and their purposes. 

College Board 

SAT 

For almost 80 years, the SAT has been a vital tool for stu- 
dents and families as they begin the college admissions 
process and has helped admissions officers make fair and 
informed college admissions decisions. The most widely 
used and most rigorously researched college admissions 
test in history, the SAT Reasoning Test is a measure of 
the critical thinking skills students need for academic 
success in college. The SAT is strongly aligned with cur- 
rent curriculum and institutional practices in high school 
and college and assesses how well students analyze and 
solve problems — skills that are needed in college. 

Typically taken by high school juniors and seniors, the 
SAT is a three-hour-and-45-minute test that measures 
critical reading, mathematical reasoning and writing 
skills that students have developed over time and that 
they need to be successful in college. The SAT consists 
of three sections: Critical Reading, which has sentence 
completion and passage-based questions; Mathematics, 
which is based on the math that college-bound students 
typically learn during their first three years of high 
school; and Writing, which has multiple-choice questions 



and a written essay. By including the measure of writ- 
ing skills, the SAT reinforces the importance of writing 
throughout a student’s education and helps colleges make 
better admissions and placement decisions. The SAT 
with the writing section was administered for the first 
time in March 2005 for the class of 2006. 

ACCUPLACER 

Colleges use ACCUPLACER test scores in conjunction 
with other information about a student’s academic 
background to help determine the most appropriate 
course placements for students. 

Most institutions have a number of developmental 
courses as well as college -level courses in which entering 
students are enrolled. Institutions have developed cut 
scores that are used to determine in which courses stu- 
dents can enroll. Students who do not attain the required 
score for placement into a college -level course are placed 
into a developmental course. 

Many colleges have multiple levels of developmental 
courses, especially in mathematics. Because ACC- 
UPLACER scores are used to place students into 
developmental courses as well as college -level courses, the 
tests’ content areas cover a broad range of skills, not just 
prerequisite skills for entry into college-level courses. 

There are three ACCUPLACER math tests: Arithmetic, 
Elementary Algebra and College Level Math. Most col- 
leges use a combination of scores to place students into 
the various math courses offered on their campus. 

■ Students with low scores on the Arithmetic and 
Elementary Algebra tests are usually placed in a low- 
level developmental math class that deals with basic 
concepts of arithmetic. 

■ Students with scores in the middle to high range of 
Arithmetic but low scores on Elementary Algebra are 
usually placed into an Elementary Algebra class that 
deals with high school Algebra I concepts. 
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■ Students with scores in the middle to high range on 
the Elementary Algebra test but low scores on the 
College Level Math test are generally placed into an 
Intermediate Algebra class that deals with high school 
Algebra II concepts. 

■ Students with scores in the middle range of College 
Level Math are generally placed into college-level 
math courses. Higher College Level Math test scores 
are used to place students into higher-level math 
courses depending on the prerequisite skills deemed 
necessary for success in the course. 

Some colleges may use only Arithmetic and/or 
Elementary Algebra test scores for placement into 
some college-level courses such as Math for Liberal Arts 
Majors because the prerequisite skills for success are 
sufficiently measured by Arithmetic and/or Elementary 
Algebra tests. 

The ACCUPLACER is a computer-adaptive test . 30 In 
an adaptive test there is a pool of test items that have 
been calibrated for difficulty and content. The first 
question presented is of average difficulty and is chosen 
randomly from several starter questions of the same level 



of difficulty. If a student answers the question incorrectly, 
the next question to be administered is chosen from a 
group of easier questions, whereas a correct answer will 
cause the next question to be somewhat more difficult. 

The test delivery system continues this process through- 
out the test, choosing the next question that is expected 
to yield the most information about examinees. To 
ensure that the test is balanced in content and that the 
kinds of questions presented do not differ greatly from 
one student to another except in difficulty, a number of 
constraints are built into the program. These constraints 
guide the selection of questions to be administered so 
that balance is achieved regardless of the skill level of 
the individual. Although fewer questions are presented 
for each test than would be given in a paper-and-pencil 
test, great accuracy is maintained with challenging tests 
corresponding to each student’s skill level. Because of the 
adaptive nature of the tests, the questions presented on 
successive tests will vary, greatly reducing the effects of 
repeated practice on the tests. The elimination of 
repeated questions will be even more marked as time 
passes and the student’s skills change. 
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